CN112578716A - Humanoid robot control system for shoe and clothes display - Google Patents
Humanoid robot control system for shoe and clothes display Download PDFInfo
- Publication number
- CN112578716A CN112578716A CN202011536802.6A CN202011536802A CN112578716A CN 112578716 A CN112578716 A CN 112578716A CN 202011536802 A CN202011536802 A CN 202011536802A CN 112578716 A CN112578716 A CN 112578716A
- Authority
- CN
- China
- Prior art keywords
- user
- information
- communicated
- control module
- chassis control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000007 visual effect Effects 0.000 claims abstract description 52
- 230000002452 interceptive effect Effects 0.000 claims abstract description 12
- 238000000034 method Methods 0.000 claims description 31
- 238000004891 communication Methods 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 23
- 238000004458 analytical method Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 13
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 10
- 230000003993 interaction Effects 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 238000005516 engineering process Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000007726 management method Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 6
- 210000000056 organ Anatomy 0.000 claims description 6
- 230000037237 body shape Effects 0.000 claims description 5
- 238000005520 cutting process Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000009286 beneficial effect Effects 0.000 claims description 4
- 238000012512 characterization method Methods 0.000 claims description 4
- 210000002364 input neuron Anatomy 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 230000008520 organization Effects 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 238000003786 synthesis reaction Methods 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 2
- 238000009826 distribution Methods 0.000 claims description 2
- 238000011478 gradient descent method Methods 0.000 claims description 2
- 238000011423 initialization method Methods 0.000 claims description 2
- 210000002569 neuron Anatomy 0.000 claims description 2
- 230000004044 response Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000011160 research Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000013480 data collection Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000011664 nicotinic acid Substances 0.000 description 2
- 238000012356 Product development Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000002567 autonomic effect Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
- G05B19/0423—Input/output
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/24—Pc safety
- G05B2219/24215—Scada supervisory control and data acquisition
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a humanoid robot control system for shoe and clothes display, which comprises a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC (personal computer) end software system, wherein the power supply module is connected with the chassis control module; the power supply module comprises a power supply and a power supply electric quantity display device; the chassis control module comprises an industrial personal computer, a chassis control panel, a laser radar, a lamp strip controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves, an IR receiver, a motor driver and a motor; the network control module comprises a switch and a router; the visual voice control module comprises a camera, a core board, a four-microphone bottom board, a microphone array, a power amplifier board, a loudspeaker and a filter; the model motion control module comprises a controller, a stepping motor driver and a stepping motor. The system can realize autonomous navigation, path planning, obstacle avoidance and the like of the robot, and can realize dynamic and man-machine interactive shoe and clothes display, popularization and the like.
Description
Technical Field
The invention relates to the field of robot control systems, in particular to a control system of a humanoid robot applied to shoe and clothes display.
Background
The shoe and clothes field is always an industry field with extremely high product iteration updating speed, and various factors such as the body type, the favor, the aesthetic feeling, the trend and the like of a client directly influence the market feedback effect of the launching of a new product. Therefore, the method is very important for market research before new product development and sample display before mass production in the field of shoe and clothes industry. In order to obtain detailed information of customers and promote new products of shoes and clothes to become market weathervane, shoe and clothes enterprises can invest a large amount of funds to finish market research and sample display work, and even invest heavy funds to engage in professional models or participate in fashion shows. The research and the display of modern large-scale shopping malls are one of necessary war fields for large data collection and sample display of shoe and clothes enterprises. From the beginning of this century, with the development of integrated chip and microcomputer technologies, robotics and technology have been rapidly developed and become one of the representative fields of high and new technologies. The humanoid robot is a bionic robot system which is high-order, nonlinear, strong in coupling performance and incompletely constrained and simulates the structure and the function of a human body, integrates various disciplines such as electromechanical engineering, material science, sensor application, control technology, artificial intelligence and the like, is one of the hottest directions in the technical field of the bionic robot, and opens up a new space for the development of service type and display type robots.
At present, the mannequin props applied to shoe and clothes display generally adopt static mannequins, the static display props cannot completely display the design characteristics, material attributes and action sensory comfort of the shoe and clothes, aesthetic fatigue is formed for customers, and the customers cannot really express own favor and demand through static mannequins or questionnaires. Therefore, the existing research and display method of footwear in superstores gradually loses the expected effect, and it is very important to find a novel, intuitive and attractive research and display method.
Disclosure of Invention
The invention aims to provide a humanoid robot control system for shoe and clothes display, which can realize autonomous navigation, path planning, obstacle avoidance and the like of a robot, and can realize dynamic and man-machine interactive shoe and clothes display, popularization and the like.
In order to achieve the purpose, the technical scheme of the invention is as follows: a humanoid robot control system for shoe and clothes display is characterized by comprising a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC (personal computer) end software system; the power supply module comprises a power supply and a power supply electric quantity display device; the chassis control module comprises an industrial personal computer, a chassis control panel, a laser radar, a lamp strip controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves, an IR receiver, a motor driver and a motor; the network control module comprises a switch and a router; the visual voice control module comprises a camera, a core board, a four-microphone bottom board, a microphone array, a power amplifier board, a loudspeaker and a filter; the model motion control module comprises a controller, a stepping motor driver and a stepping motor; the chassis control board and the core board are connected and communicated with a PC end software system through a router.
The power supply provides electric energy for each module, the power supply electric quantity display device is connected and communicated with the chassis control panel through a coulometer, the industrial personal computer is respectively connected and communicated with the chassis control panel and the laser radar, the lamp strip controller, the anti-collision switch, the emergency stop switch, the ultrasonic wave receiver, the IR receiver and the motor driver are respectively connected and communicated with the chassis control panel, the motor is connected with and driven by a motor driver, the exchanger is respectively connected with and communicated with an industrial personal computer, a router and a chassis control panel, the camera and the core board are respectively connected and communicated with the router, the four-microphone bottom board and the power amplifier board are respectively connected and communicated with the core board, the filter and the loudspeaker are connected with the power amplifier board, the microphone array is connected with the four-microphone bottom board, the stepping motor driver is connected with the controller, the stepping motor is connected with the stepping motor driver, and the controller is connected and communicated with the chassis control panel.
The chassis control panel is for adopting STM32 chassis control panel, and/or, nuclear core plate is for adopting RK3288 nuclear core plate.
The coulometer, the lamp belt controller and the chassis control panel are connected and communicated through an RS485 serial port; the industrial personal computer is connected and communicated with the laser radar and the switch through the network port equipment; the router is connected with the core board and the PC end software system through the internet access equipment for communication; the industrial personal computer is connected and communicated with the camera through a USB interface; the chassis control panel is connected and communicated with the anti-collision switch, the emergency stop switch and the controller through GPIO ports; the industrial personal computer is connected with the chassis control panel through the CAN for communication; the ultrasonic waves are connected and communicated with the chassis control panel through TTL; the motor driver is connected with the chassis control panel through the CAN for communication; the core board is connected and communicated with the four-microphone bottom board through an audio port and a USB interface; and/or; the core board is connected with the power amplifier board through the audio port for communication.
The operation mode of the humanoid robot control system comprises a navigation mode and a service mode, and the mode switching is controlled by a PC end software system; in a navigation mode, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of the model, and the visual voice module works to realize voice propaganda and popularization of products; under the business mode, the visual voice module works to realize visual data acquisition and man-machine voice interaction communication; the PC side software system comprises a user UI interface and a database management, wherein the main functions of the user UI interface comprise drawing construction, navigation queuing, map and position information display, model motion control and service mode point configuration and display, and the main functions of the database management comprise visual identification information display, navigation points and service mode points.
The method for realizing man-machine voice interactive communication in the visual voice module is that,
1) after the user voice is output, the visual voice module acquires the user voice, and the input voice with higher quality is acquired through noise suppression, echo cancellation, sound source positioning and far-field pickup technology when the user voice is acquired;
2) the visual voice module analyzes the keywords of the user voice to obtain language keywords and/or keywords related to the user;
3) performing interactive language processing on the language keywords, specifically performing interactive language processing by acquiring a knowledge base of a pre-established and stored natural language semantic template to obtain a first answer sentence content, wherein the knowledge base adopts a method of manually collecting, sorting and storing data and adopts a machine active learning method to accept user training and collect useful information of an organization to enrich the knowledge base;
performing user analysis on the keywords related to the user, specifically, performing user analysis by acquiring information of a pre-established user library to obtain user information, adding the user information into the user library, and sorting the user information to obtain answer sentence content II;
meanwhile, the service system of the visual voice module obtains a result through keyword analysis to perform product retrieval, specifically obtains a product retrieval result through obtaining information of a pre-established product library and a user library and arranges the product retrieval result to obtain a third answer sentence content;
4) the visual voice module carries out sentence synthesis processing on the first answer sentence content, the second answer sentence content and the third answer sentence content to form a complete answer sentence;
5) and outputting the complete answer sentence.
The visual voice module is used for collecting visual data and estimating and analyzing the appearance and body type of a user, specifically, a face recognition system and a body type recognition system in the visual voice module are used for estimating and analyzing to obtain user information comprising user face information, user gender information and user body type information, and the user information is added to a pre-established user library and/or is supplied for user analysis during man-machine voice interaction communication; the face recognition system comprises four steps, namely face image acquisition and face detection, face image preprocessing, face image feature extraction and face image matching and recognition.
The human face image acquisition and the human face detection in the human face detection adopt an Adaboost learning algorithm; the face image feature extraction adopts a knowledge-based characterization method, and obtains feature data which is beneficial to face classification according to shape description of face organs and distance characteristics between the face organs; the face image matching and recognition is implemented by setting a threshold, comparing the face features to be recognized with face feature templates obtained in a pre-established user library, outputting a matching result when the similarity exceeds the set threshold, and estimating gender information and age information of the recognized person according to the matching result.
The visual voice module adopts a convolutional neural network model to carry out gender prediction classification and age prediction classification, gender prediction is taken as a classification problem in the gender prediction classification, a human face detected by a camera is taken as the input of a gender prediction network, the CNN network is utilized to extract features, the output layer of the gender prediction network is of a softmax type, and 2 nodes represent two categories of males and females as the output; the age prediction problem is defined as a classification problem in the age prediction, 0-100 years are divided into N age groups, and the age prediction network corresponds to N nodes at the last layer of softmax to represent the age range.
The body type recognition system divides the body type of a human body into five grades of a thin body type Y, a thinner body type YA, a common type A, a slightly fat type AB and a fat type B, extracts various image information characteristics according to the standing characteristics of the human body, carries out body type recognition by constructing a BP-Adaboost model, takes a BP neural network as a weak classifier, repeatedly trains the BP neural network to predict sample output, obtains a strong classifier consisting of a plurality of BP neural network weak classifiers by an Adaboost algorithm, and finally carries out body type grading and grading through threshold setting.
The calculation method comprises extracting height information and body width information after processing the image obtained by the camera, calculating grade and comparing to obtain corresponding body type grade,
wherein H represents the height of the body type obtained after image processing, and W represents the body type obtained after image processingA width; alpha is alpha1、α2、α3、α4Representing different thresholds at different body type levels.
By adopting the technical scheme, the invention has the beneficial effects that: the humanoid robot control system comprises a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC (personal computer) end software system, wherein multiple operation modes can be realized through the application of equipment devices of the modules and the arrangement of a connection communication structure between equipment periods, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of the model, and the visual voice module works to realize voice propaganda and popularization of products, visual data acquisition, human face recognition, body type recognition, estimation and analysis and man-machine voice interaction communication; the PC side software system comprises a user UI interface and database management, and can perform operations of parameter adjustment, control, operation mode change and the like of the robot. In conclusion, the control system structure of the invention can enable the robot to achieve more humanized high-automation and high-intelligence application of movement, interaction and propaganda and promotion, can be applied to big data collection, market research and product display and promotion, and has good application performance, thereby better realizing the aim and effect of the invention.
Drawings
FIG. 1 is a block diagram of the modular construction of a footwear-oriented display humanoid robot control system in accordance with the present invention;
FIG. 2 is a block diagram of the control system of the humanoid robot for shoes and clothes display;
FIG. 3 is a block diagram of the model motion control module according to the present invention;
FIG. 4 is a block diagram of the construction of the operation mode of the humanoid robot control system for shoes and clothes display according to the present invention;
FIG. 5 is a block diagram of a PC side software system according to the present invention;
FIG. 6 is a flow chart of human-computer voice interactive communication in a visual-voice module according to the present invention;
fig. 7 is a schematic diagram of a convolutional neural network structure according to the present invention.
Detailed Description
In order to further explain the technical solution of the present invention, the present invention is explained in detail by the following specific examples.
The embodiment discloses a humanoid robot control system towards shoes and clothes show, as shown in fig. 1, including power module 1, chassis control module 2, network control module 3, model motion control module 4 and vision voice module 5, still including PC end software system 6, power supply 1 provides the electric energy for each module, chassis control module 2 works and realizes the autonomic navigation of humanoid robot, plans the route, keeps away the barrier, the dynamic show of model is realized in the work of model motion control module 3, the pronunciation propaganda of vision voice module 5 work realization product is promoted and is realized that visual data gathers and carry out face identification and size discernment estimation analysis and human-computer voice interaction and communicate, below combines the equipment device structure overall arrangement and the relation of connection that each module contained to describe in detail with fig. 2.
As shown in fig. 2, the power module 1 includes a power supply and a power supply capacity display device; the chassis control module 2 comprises an industrial personal computer, a chassis control panel (the embodiment adopts an STM32 chassis control panel), a laser radar, a lamp strip controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves (4 arranged), an IR receiver, a motor driver and motors (in the embodiment, the number of the motor driver and the number of the motor are respectively 2, and the chassis of the robot adopts walking wheels which are independently suspended and driven and work in cooperation with universal wheels); the network control module 3 comprises a switch and a router; the vision voice control module 5 comprises a camera, a core board (the RK3288 core board is adopted in this embodiment), a four-microphone base board, a microphone array, a power amplifier board, a loudspeaker (2 loudspeakers are arranged), and a filter, wherein the power supply is configured with a power supply adaptive to electric quantity according to the working condition; the model motion control module 4 comprises a controller, a stepping motor driver and a stepping motor.
The connection communication relationship of the individual device in this embodiment is specifically as follows: the chassis control board and the core board are connected and communicated with a PC end software system through a router. The power supply electric quantity display device is connected and communicated with the chassis control panel through a coulometer, the industrial personal computer is respectively connected and communicated with the chassis control panel and the laser radar, the lamp belt controller, the anti-collision switch, the emergency stop switch, the ultrasonic wave receiver, the IR receiver and the motor driver are respectively connected with and communicated with the chassis control panel, the motor is connected with and driven by a motor driver, the exchanger is respectively connected with and communicated with an industrial personal computer, a router and a chassis control panel, the camera and the core board are respectively connected and communicated with the router, the four-microphone bottom board and the power amplifier board are respectively connected and communicated with the core board, the filter and the loudspeaker are connected with the power amplifier board, the microphone array is connected with the four-microphone bottom board, the stepping motor driver is connected with the controller, the stepping motor is connected with the stepping motor driver, and the controller is connected and communicated with the chassis control panel. In the embodiment, the coulometer and the lamp belt controller are connected and communicated with the chassis control panel through an RS485 serial port; the industrial personal computer is connected and communicated with the laser radar and the switch through the network port equipment; the router is connected with the core board and the PC end software system through the internet access equipment for communication; the industrial personal computer is connected and communicated with the camera through a USB interface; the chassis control panel is connected and communicated with the anti-collision switch, the emergency stop switch and the controller through GPIO ports; the industrial personal computer is connected with the chassis control panel through the CAN for communication; the ultrasonic waves are connected and communicated with the chassis control panel through TTL; the motor driver is connected with the chassis control panel through the CAN for communication; the core board is connected and communicated with the four-microphone bottom board through an audio port and a USB interface; the core board is connected with the power amplifier board through the audio port for communication. The connection mode of the controller and the driver in the model motion control module 4 is as follows; the PUL + (pulse +) and DIR + (direction +) terminals of the stepping motor driver are connected with +5V, and the PUL- (pulse-) terminals are connected with the PUL- (pulse-) and DIR- (direction-) terminals of the controller are connected with the DIR- (direction-) of the controller. The power VCC (positive power) of the stepper motor driver is connected with the positive power, and the power GND (negative power) is connected with the negative power, as shown in FIG. 3.
The control system structure of the invention can enable the robot to achieve more humanized movement, interaction, high automation and high intelligence application of propaganda and popularization.
In this embodiment, an operation mode of the humanoid robot control system of the present invention is disclosed, which includes a navigation mode and a service mode, as shown in fig. 4 and 5, the switching of the modes is controlled by a PC-side software system 6, the robot can also control its movement and stop movement through the PC-side software system, and the stop movement can also be controlled manually; in a navigation mode, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of the model, and the visual voice module works to realize voice propaganda and popularization of products; under the business mode, the visual voice module works to realize visual data acquisition and man-machine voice interaction communication and provide directional guidance for popularizing products according to the requirements of customers; the PC side software system comprises a user UI interface and a database management, wherein the main functions of the user UI interface comprise drawing construction, navigation queuing, map and position information display, model motion control and service mode point configuration and display, and the main functions of the database management comprise visual identification information display, navigation points and service mode points, namely the functions of visual identification information storage, data classification, data display and the like. The PC end software system issues general instructions, communicates with the chassis control panel through zmq, and communicates with the visual recognition system and the voice recognition system through TCP.
The autonomous navigation function is one of core technologies of the shoe and clothes display humanoid robot running in an indoor environment, and the autonomous navigation function realizes sensing of environmental information and self state and autonomous movement of autonomous obstacle avoidance. Information provided by a laser radar, sensing and the like is integrated to form unified representation of an external environment, fusion of various information can play a complementary role, after the real-time performance and redundancy of the information are guaranteed, environmental characteristics are reflected, correct judgment and decision are made, a robot is precisely positioned by utilizing an extended Kalman filtering algorithm to fuse laser and other sensor data, a local map is generated by carrying out data acquisition and processing on the environmental information through the carried laser radar, a map service is started to be updated in the global map, the local map is continuously scanned and updated in the environment in a circulating movement mode until each updating information of the local information is contained in the global map, and map construction is completed.
The navigation is divided into global navigation and local navigation, and algorithms adopted in the global navigation include an A-x algorithm and a Dijkstra algorithm, and the A-x algorithm and the Dijkstra algorithm are responsible for planning a track from an initial position to a target position. After the global path is determined, the shoe and clothes display humanoid robot adopts a track expansion method or a dynamic window method to adopt local path planning, namely local path planning, in the real-time navigation process, and is responsible for specific speed issuing and obstacle avoidance.
The method for implementing the man-machine voice interactive communication in the visual voice module is that, as shown in figure 6,
1) after the user voice is output, the visual voice module acquires the user voice, and the input voice with higher quality is acquired through noise suppression, echo cancellation, sound source positioning and far-field pickup technology when the user voice is acquired;
2) the visual voice module analyzes the keywords of the user voice to obtain language keywords and/or keywords related to the user;
3) performing interactive language processing on the language keywords, specifically performing interactive language processing by acquiring a knowledge base of a pre-established and stored natural language semantic template to obtain a first answer sentence content, wherein the knowledge base adopts a method of manually collecting, sorting and storing data and adopts a machine active learning method to accept user training and collect useful information of an organization to enrich the knowledge base;
performing user analysis on the keywords related to the user, specifically, performing user analysis by acquiring information of a pre-established user library to obtain user information, adding the user information into the user library, and sorting the user information to obtain answer sentence content II;
meanwhile, the service system of the visual voice module obtains a result through keyword analysis to perform product retrieval, specifically obtains a product retrieval result through obtaining information of a pre-established product library and a user library and arranges the product retrieval result to obtain a third answer sentence content;
4) the visual voice module carries out sentence synthesis processing on the first answer sentence content, the second answer sentence content and the third answer sentence content to form a complete answer sentence;
5) and outputting the complete answer sentence.
The visual voice module is used for collecting visual data and estimating and analyzing the appearance and body type of a user, specifically, a face recognition system and a body type recognition system in the visual voice module are used for estimating and analyzing to obtain user information comprising user face information, user gender information and user body type information, and the user information is added to a pre-established user library and/or is supplied for user analysis during man-machine voice interaction communication; the face recognition system comprises four steps, namely face image acquisition and face detection, face image preprocessing, face image feature extraction and face image matching and recognition. In conclusion, the visual identification is used for collecting the face information, the gender information and the body shape information of the customer, estimating the basic information of the customer and finishing the recommendation of corresponding footwear and clothing products according to different people.
The face image acquisition and the face detection in the face detection adopt an Adaboost learning algorithm, which is a method for classification, the algorithm selects some rectangular features (weak classifiers) capable of representing the face most, the weak classifiers are constructed into a strong classifier according to a weighted voting mode, and then a plurality of strong classifiers obtained by training are connected in series to form a cascade-structured stacked classifier; the face image preprocessing is a process of processing an image based on a face detection result and finally serving for feature extraction, and the original image acquired by the system is subjected to image preprocessing such as gray level correction, noise filtration and the like due to the limitation and random interference of various conditions; the human face image feature extraction is also called human face characterization, which is a process for carrying out feature modeling on a human face, and adopts a knowledge-based characterization method to obtain feature data which is beneficial to human face classification according to shape description of human face organs and distance characteristics between the human face organs; the face image matching and recognition is implemented by setting a threshold, comparing the face features to be recognized with face feature templates obtained in a pre-established user library, outputting a matching result when the similarity exceeds the set threshold, and estimating gender information and age information of the recognized person according to the matching result.
The visual voice module adopts a convolutional neural network model to carry out gender prediction classification and age prediction classification, gender prediction is taken as a classification problem in the gender prediction classification, a human face detected by a camera is taken as the input of a gender prediction network, the CNN network is utilized to extract features, the output layer of the gender prediction network is of a softmax type, and 2 nodes represent two categories of 'male' and 'female' as outputs; the age prediction problem is defined as a classification problem in the age prediction, 0-100 years old is divided into N age groups, such as one group with the age between 0-2, another group with the age between 4-6, and so on. [ (0-2), (4-6), (8-12), (15-20), (25-32), (38-43), (48-53), (60-100) ], the age prediction network corresponds to N (8) nodes at the last level of softmax indicating the age range. The following specifically discloses a method for gender prediction classification and age prediction classification by using a convolutional neural network model and various parameter settings in the method, which can achieve a better function effect of age prediction and body type prediction applied to a humanoid robot control system for shoe and clothing display.
Firstly, in the input process, a given image is zoomed to M x M proportion, then 3-channel color image processing is carried out, if the image does not accord with the size, the redundant frame needs to be cut, and the cutting process is that the center of the image is cut to four sides. Specifically, the convolutional neural network processing is performed, and the present invention is described with respect to using 3 convolutional layers and 2 fully-connected layers, as shown in fig. 7.
First convolutional layer: 96 convolution kernels are adopted, the number of parameters of each convolution kernel is 3 × 7, the ReLU is adopted as an activation function, the maximum overlapping pooling is adopted as the pooling, the pooled size is selected to be 3 × 3, the strains is selected to be 2, and the convolution step size is 4. The layer is then subsequently normalized to the local response.
Where α represents the output after convolutional layer, i.e. a four-dimensional number
[batch,height,width,channel]Batch represents the number of batches, height represents the picture height, width represents the picture width, and channel is the number of channels.
Represents a position [ a, b, c, d ] in this output structure]I.e. the point under the d-th channel of the a-th graph where the height is b and the width is c.
A second convolutional layer: the input of the second layer processes the processed single-channel picture again, 256 filters are selected, the size of the filter is 5^2, and the convolution step length is 1.
A third convolutional layer: the number of the filters is 384, and the size of the convolution kernel is 3^ 2.
For a fully connected layer: the number of the neurons in each layer is selected to be 2^ 9.
An output layer: for gender, the classification is two, and the number of input neurons is 2; for age 8 classes, the number of input neurons is 8.
Training process:
(1) initializing parameters: the weight initialization method uses a gaussian positive-phase distribution with a standard deviation of 0.01 and a mean of 0.
(2) Network training: dropout is used to limit the overfitting. The Drop out ratio is 0.5, and data expansion is performed by inputting M × M pictures and then cropping.
(3) The training method adopts a random gradient descent method, the min-batch size is selected to be 50, the learning rate is 0.001, and then the learning rate is adjusted to be 0.0001 after the iteration is carried out to 10000 times.
(4) And (4) predicting the result: the prediction method is characterized in that 256 × 256 pictures are input, then the pictures are processed for multiple times, and finally prediction results are averaged.
The body type recognition system divides the body type of a human body into five grades of a thin body type Y, a thinner body type YA, a common type A, a slightly fat type AB and a fat type B, extracts various image information characteristics according to the standing characteristics of the human body, carries out body type recognition by constructing a BP-Adaboost model, takes a BP neural network as a weak classifier, repeatedly trains the BP neural network to predict sample output, obtains a strong classifier consisting of a plurality of BP neural network weak classifiers by an Adaboost algorithm, and finally carries out body type grading and grading through threshold setting.
The calculation method comprises extracting height information and body width information after processing the image obtained by the camera, calculating grade and comparing to obtain corresponding body type grade,
wherein H represents the height of the body shape obtained after image processing, and W represents the width of the body shape obtained after image processing; alpha is alpha1、α2、α3、α4Representing different thresholds at different body type levels.
The above embodiments and drawings are not intended to limit the form and style of the present invention, and any suitable changes or modifications thereof by those skilled in the art should be considered as not departing from the scope of the present invention.
Claims (10)
1. A humanoid robot control system for shoe and clothes display is characterized by comprising a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC (personal computer) end software system; the power supply module comprises a power supply and a power supply electric quantity display device; the chassis control module comprises an industrial personal computer, a chassis control panel, a laser radar, a lamp strip controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves, an IR receiver, a motor driver and a motor; the network control module comprises a switch and a router; the visual voice control module comprises a camera, a core board, a four-microphone bottom board, a microphone array, a power amplifier board, a loudspeaker and a filter; the model motion control module comprises a controller, a stepping motor driver and a stepping motor; the chassis control board and the core board are connected and communicated with a PC end software system through a router;
the power supply provides electric energy for each module, the power supply electric quantity display device is connected and communicated with the chassis control panel through a coulometer, the industrial personal computer is respectively connected and communicated with the chassis control panel and the laser radar, the lamp strip controller, the anti-collision switch, the emergency stop switch, the ultrasonic wave receiver, the IR receiver and the motor driver are respectively connected and communicated with the chassis control panel, the motor is connected with and driven by a motor driver, the exchanger is respectively connected with and communicated with an industrial personal computer, a router and a chassis control panel, the camera and the core board are respectively connected and communicated with the router, the four-microphone bottom board and the power amplifier board are respectively connected and communicated with the core board, the filter and the loudspeaker are connected with the power amplifier board, the microphone array is connected with the four-microphone bottom board, the stepping motor driver is connected with the controller, the stepping motor is connected with the stepping motor driver, and the controller is connected and communicated with the chassis control panel.
2. The humanoid robot control system for shoe and clothes display of claim 1, wherein the chassis control panel is a STM32 chassis control panel, and/or the core panel is a RK3288 core panel; the coulometer, the lamp belt controller and the chassis control panel are connected and communicated through an RS485 serial port; the industrial personal computer is connected and communicated with the laser radar and the switch through the network port equipment; the router is connected with the core board and the PC end software system through the internet access equipment for communication; the industrial personal computer is connected and communicated with the camera through a USB interface; the chassis control panel is connected and communicated with the anti-collision switch, the emergency stop switch and the controller through GPIO ports; the industrial personal computer is connected with the chassis control panel through the CAN for communication; the ultrasonic waves are connected and communicated with the chassis control panel through TTL; the motor driver is connected with the chassis control panel through the CAN for communication; the core board is connected and communicated with the four-microphone bottom board through an audio port and a USB interface; and/or; the core board is connected with the power amplifier board through the audio port for communication.
3. The humanoid robot control system for shoe and clothes display of claim 1, wherein the operation mode of the humanoid robot control system comprises a navigation mode and a service mode, and the switching of the modes is controlled by a PC-side software system; in a navigation mode, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of the model, and the visual voice module works to realize voice propaganda and popularization of products; under the business mode, the visual voice module works to realize visual data acquisition and man-machine voice interaction communication; the PC side software system comprises a user UI interface and a database management, wherein the main functions of the user UI interface comprise drawing construction, navigation queuing, map and position information display, model motion control and service mode point configuration and display, and the main functions of the database management comprise visual identification information display, navigation points and service mode points.
4. The humanoid robot control system for footwear display of claim 3, wherein the method for implementing human-computer voice interactive communication in the visual voice module is such that,
1) after the user voice is output, the visual voice module acquires the user voice, and the input voice with higher quality is acquired through noise suppression, echo cancellation, sound source positioning and far-field pickup technology when the user voice is acquired;
2) the visual voice module analyzes the keywords of the user voice to obtain language keywords and/or keywords related to the user;
3) performing interactive language processing on the language keywords, specifically performing interactive language processing by acquiring a knowledge base of a pre-established and stored natural language semantic template to obtain a first answer sentence content, wherein the knowledge base adopts a method of manually collecting, sorting and storing data and adopts a machine active learning method to accept user training and collect useful information of an organization to enrich the knowledge base;
performing user analysis on the keywords related to the user, specifically, performing user analysis by acquiring information of a pre-established user library to obtain user information, adding the user information into the user library, and sorting the user information to obtain answer sentence content II;
meanwhile, the service system of the visual voice module obtains a result through keyword analysis to perform product retrieval, specifically obtains a product retrieval result through obtaining information of a pre-established product library and a user library and arranges the product retrieval result to obtain a third answer sentence content;
4) the visual voice module carries out sentence synthesis processing on the first answer sentence content, the second answer sentence content and the third answer sentence content to form a complete answer sentence;
5) and outputting the complete answer sentence.
5. The humanoid robot control system for shoe and clothes display of claim 3 or 4, characterized in that, the visual voice module collects visual data for estimation and analysis of the appearance and body type of the user, specifically, the human face recognition system and the body type recognition system in the visual voice module are used for estimation and analysis to obtain user information including the user face information, the user gender information and the user body type information, and the user information is added to a pre-established user library and/or provided for user analysis during human-computer voice interaction communication; the face recognition system comprises four steps, namely face image acquisition and face detection, face image preprocessing, face image feature extraction and face image matching and recognition.
6. The humanoid robot control system for shoes and clothes exhibition as claimed in claim 5, wherein the human face detection in the human face image acquisition and human face detection adopts Adaboost learning algorithm; the face image feature extraction adopts a knowledge-based characterization method, and obtains feature data which is beneficial to face classification according to shape description of face organs and distance characteristics between the face organs; the face image matching and recognition is implemented by setting a threshold, comparing the face features to be recognized with face feature templates obtained in a pre-established user library, outputting a matching result when the similarity exceeds the set threshold, and estimating gender information and age information of the recognized person according to the matching result.
7. The humanoid robot control system for shoes and clothes display of claim 6, wherein the visual speech module adopts a convolutional neural network model to perform gender prediction classification and age prediction classification, wherein gender prediction is taken as a classification problem in the gender prediction classification, a human face detected by a camera is taken as an input of the gender prediction network, characteristics are extracted by using a CNN network, an output layer of the gender prediction network is of a softmax type, and 2 nodes represent two categories of males and females as outputs; the age prediction problem is defined as a classification problem in the age prediction, 0-100 years are divided into N age groups, and the age prediction network corresponds to N nodes at the last layer of softmax to represent the age range.
8. The humanoid robot control system for shoe and clothes display of claim 5, wherein the body type recognition system divides the human body into five grades of a thin body type Y, a thinner body type YA, a normal type A, a slightly fat type AB and a fat type B, extracts a plurality of image information characteristics according to the standing characteristics of the human body, performs body type recognition by constructing a BP-Adaboost model, uses a BP neural network as a weak classifier, repeatedly trains the BP neural network to predict sample output, obtains a strong classifier composed of a plurality of BP neural network weak classifiers by an Adaboost algorithm, and finally performs body type grading and grading by setting a threshold value.
9. The humanoid robot control system for shoes and clothes display of claim 8, characterized in that the calculation method is to extract height information and body width information after processing the image obtained by the camera, and to obtain the corresponding human body type grade by calculating, grading, comparing and judging,
wherein H represents the height of the body shape obtained after image processing, and W represents the width of the body shape obtained after image processing; alpha is alpha1、α2、α3、α4Representing different thresholds at different body type levels.
10. The humanoid robot control system for footwear-oriented display of claim 7, wherein 0-100 years old is divided into 8 age groups; adopting a convolutional neural network model to carry out gender prediction classification and age prediction classification, adopting 3 convolutional layers and 2 full-connection layers, firstly, scaling a given image to M x M proportion in an input process, then carrying out 3-channel color image processing, if the image does not accord with the size, cutting redundant frames, and cutting the image from the center to four sides in a cutting process;
first convolutional layer: using 96 convolution kernels, 3 x 7 parameters per convolution kernel, ReLU for the activation function, max overlap pooling for pooling, 3 x 3 for pooled size, 2 for ranks, 4 for convolution step size, then subsequently normalizing the partial responses to the layer,
wherein alpha represents the output result after the convolutional layer,
represents a position [ a, b, c, d ] in this output structure]The height under the d channel of the a-th graph is a point with the b width being c;
a second convolutional layer: the input of the second layer processes the processed single-channel picture again, 256 filters are selected, the size of the filter is 5^2, and the convolution step length is 1;
a third convolutional layer: the number of the filters is 384, and the size of a convolution kernel is 3^ 2;
for a fully connected layer: selecting 2^9 as the number of the neurons in each layer;
an output layer: for gender prediction, the method is classified into two categories, and the number of input neurons is 2; for age prediction, 8 classes are used, and the number of input neurons is 8;
in the model training process, firstly, parameters are initialized, and a weight initialization method adopts Gaussian distribution with a standard deviation of 0.01 and a mean value of 0; then, network training is carried out, dropout is adopted to limit overfitting, Drop out proportion is 0.5, data expansion is carried out by inputting M pictures, then cutting is carried out, a random gradient descent method is adopted in the training method, the size of min-batch is selected to be 50, the learning rate is 0.001, and then when iteration is carried out for 10000 times, the learning rate is adjusted to be 0.0001; and finally, predicting the result, wherein the prediction method adopts the input of a 256 × 256 picture, and the prediction result is averaged by carrying out multiple times of processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536802.6A CN112578716B (en) | 2020-12-23 | Humanoid robot control system for shoe-suit display |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536802.6A CN112578716B (en) | 2020-12-23 | Humanoid robot control system for shoe-suit display |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112578716A true CN112578716A (en) | 2021-03-30 |
| CN112578716B CN112578716B (en) | 2024-07-02 |
Family
ID=
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201105467A (en) * | 2009-08-04 | 2011-02-16 | Mirle Automation Corp | Robotic fashion model |
| CN106346487A (en) * | 2016-08-25 | 2017-01-25 | 威仔软件科技（苏州）有限公司 | Interactive VR sand table show robot |
| CN206085055U (en) * | 2016-09-28 | 2017-04-12 | 深圳维周机器人科技有限公司 | Intelligence control system of model robot |
| CN110403409A (en) * | 2019-07-18 | 2019-11-05 | 王东 | A kind of robot and merchandise display method |
| CN211137173U (en) * | 2019-12-19 | 2020-07-31 | 大连雨林信息***有限公司 | Robot for clothing display |
| CN213843825U (en) * | 2020-12-23 | 2021-07-30 | 福建（泉州）哈工大工程技术研究院 | Humanoid robot control system for shoe and clothes display |
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201105467A (en) * | 2009-08-04 | 2011-02-16 | Mirle Automation Corp | Robotic fashion model |
| CN106346487A (en) * | 2016-08-25 | 2017-01-25 | 威仔软件科技（苏州）有限公司 | Interactive VR sand table show robot |
| CN206085055U (en) * | 2016-09-28 | 2017-04-12 | 深圳维周机器人科技有限公司 | Intelligence control system of model robot |
| CN110403409A (en) * | 2019-07-18 | 2019-11-05 | 王东 | A kind of robot and merchandise display method |
| CN211137173U (en) * | 2019-12-19 | 2020-07-31 | 大连雨林信息***有限公司 | Robot for clothing display |
| CN213843825U (en) * | 2020-12-23 | 2021-07-30 | 福建（泉州）哈工大工程技术研究院 | Humanoid robot control system for shoe and clothes display |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1406135B1 (en) | Man-machine interface unit control method; robot apparatus; and its action control method | |
| Mao et al. | Learning salient features for speech emotion recognition using convolutional neural networks | |
| US7379568B2 (en) | Weak hypothesis generation apparatus and method, learning apparatus and method, detection apparatus and method, facial expression learning apparatus and method, facial expression recognition apparatus and method, and robot apparatus | |
| US6353814B1 (en) | Developmental learning machine and method | |
| JP3529049B2 (en) | Learning device, learning method, and robot device | |
| Chen et al. | Adaptive feature selection-based AdaBoost-KNN with direct optimization for dynamic emotion recognition in human–robot interaction | |
| CN111291604A (en) | Face attribute identification method, device, storage medium and processor | |
| CN103413113A (en) | Intelligent emotional interaction method for service robot | |
| CN104463191A (en) | Robot visual processing method based on attention mechanism | |
| CN110781829A (en) | Light-weight deep learning intelligent business hall face recognition method | |
| CN104463100A (en) | Intelligent wheelchair man-machine interaction system and method based on facial expression recognition mode | |
| CN112068555A (en) | Voice control type mobile robot based on semantic SLAM method | |
| CN114724224A (en) | Multi-mode emotion recognition method for medical care robot | |
| Chen et al. | Emotion recognition and understanding for emotional human-robot interaction systems | |
| CN112070010A (en) | Pedestrian re-recognition method combining multi-loss dynamic training strategy to enhance local feature learning | |
| Tang et al. | Facial expression recognition using AAM and local facial features | |
| CN114202792A (en) | Face dynamic expression recognition method based on end-to-end convolutional neural network | |
| Wu et al. | From members to teams to committee-a robust approach to gestural and multimodal recognition | |
| CN213843825U (en) | Humanoid robot control system for shoe and clothes display | |
| CN112578716A (en) | Humanoid robot control system for shoe and clothes display | |
| CN112578716B (en) | Humanoid robot control system for shoe-suit display | |
| Lombardi | A survey of automatic lip reading approaches | |
| Kim et al. | Facial emotional expression recognition with soft computing techniques | |
| Weng et al. | The developmental approach to artificial intelligence: Concepts, developmental algorithms and experimental results | |
| Lu et al. | Deep neural networks for head pose classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: 362000 Building 9, software park, Fengze District, Quanzhou City, Fujian Province Applicant after: Fujian (Quanzhou) Advanced Manufacturing Technology Research Institute Address before: 362000 Building 9, software park, Fengze District, Quanzhou City, Fujian Province Applicant before: FUJIAN (QUANZHOU) HIT Research Institute OF ENGINEERING AND TECHNOLOGY Country or region before: China |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant |