CN112578716B - Humanoid robot control system for shoe-suit display - Google Patents
Humanoid robot control system for shoe-suit display Download PDFInfo
- Publication number
- CN112578716B CN112578716B CN202011536802.6A CN202011536802A CN112578716B CN 112578716 B CN112578716 B CN 112578716B CN 202011536802 A CN202011536802 A CN 202011536802A CN 112578716 B CN112578716 B CN 112578716B
- Authority
- CN
- China
- Prior art keywords
- user
- board
- information
- chassis control
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000000007 visual effect Effects 0.000 claims abstract description 56
- 230000033001 locomotion Effects 0.000 claims abstract description 25
- 230000003993 interaction Effects 0.000 claims abstract description 8
- 230000003321 amplification Effects 0.000 claims abstract description 7
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 7
- 238000004891 communication Methods 0.000 claims description 39
- 238000000034 method Methods 0.000 claims description 32
- 238000012545 processing Methods 0.000 claims description 21
- 238000004458 analytical method Methods 0.000 claims description 14
- 230000002452 interceptive effect Effects 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000007726 management method Methods 0.000 claims description 7
- 210000000056 organ Anatomy 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012512 characterization method Methods 0.000 claims description 4
- 210000002364 input neuron Anatomy 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 230000009286 beneficial effect Effects 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 2
- 238000009826 distribution Methods 0.000 claims description 2
- 238000011478 gradient descent method Methods 0.000 claims description 2
- 238000011423 initialization method Methods 0.000 claims description 2
- 210000002569 neuron Anatomy 0.000 claims description 2
- 230000004044 response Effects 0.000 claims description 2
- 238000010606 normalization Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 7
- 238000011160 research Methods 0.000 description 6
- 241000209140 Triticum Species 0.000 description 4
- 235000021307 Triticum Nutrition 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000011664 nicotinic acid Substances 0.000 description 2
- 238000012356 Product development Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000007115 recruitment Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
- G05B19/0423—Input/output
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/24—Pc safety
- G05B2219/24215—Scada supervisory control and data acquisition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a humanoid robot control system for shoe and suit display, which comprises a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC end software system, wherein the power supply module is connected with the chassis control module; the power module comprises a power supply and a power supply electric quantity display device; the chassis control module comprises an industrial personal computer, a chassis control board, a laser radar, a lamp band controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves, an IR receiver, a motor driver and a motor; the network control module comprises a switch and a router; the visual voice module comprises a camera, a core board, a four-microphone base board, a microphone array, a power amplification board, a loudspeaker and a filter; the model motion control module includes a controller, a stepper motor driver, and a stepper motor. The system can realize autonomous navigation, path planning, obstacle avoidance and the like of the robot, and can realize dynamic shoe suit display, popularization and the like capable of realizing man-machine interaction.
Description
Technical Field
The invention relates to the field of robot control systems, in particular to a control system of a humanoid robot applied to shoe suit display.
Background
The shoe field is always an industry field with extremely high iterative updating speed of products, and various factors such as body type, preference, aesthetic and trend of customers directly influence the market feedback effect of the release of new products. Therefore, the method is extremely important to market research before new product development and sample display before mass production in the field of shoe industry. In order to acquire the detailed information of customers, the shoe-wear new product is pushed to become a market vane, and shoe-wear enterprises can invest a large amount of funds to complete market research and sample display work, and even invest heavy recruitment to engage in professional models or participate in fashion shows. The research and the display of the modern large-scale market are one of the necessary grounds for the collection of the large data and the sample display of shoe and clothing enterprises. From this century, robot science and technology have been rapidly developed with the development of integrated chips and microcomputer technology, and have become one of the representative fields of high and new technologies. The human-shaped robot is a high-order, nonlinear, strong-coupling and incompletely constrained bionic robot system simulating the structure and the function of a human body, integrates various subjects such as electromechanical engineering, material science, sensor application, control technology, artificial intelligence and the like, is one of the hottest directions in the technical field of the bionic robot, and opens up a new space for the development of service-type and display-type robots.
At present, a mannequin prop applied to shoe wear display generally adopts a static mannequin, the static display prop cannot fully display the design characteristics, material properties and action sensory comfort of the shoe wear, aesthetic fatigue is formed for a customer, and the customer cannot faithfully express the preference and the demand of the customer through a static mannequin or a questionnaire. Therefore, the existing market shoes and clothing research and display modes gradually lose the expected effect, and it is extremely important to seek a novel, visual and attractive research and display mode.
Disclosure of Invention
The invention aims to provide a humanoid robot control system for shoe-suit display, which can realize autonomous navigation, path planning, obstacle avoidance and the like of a robot, and can realize dynamic shoe-suit display, popularization and the like capable of realizing man-machine interaction.
In order to achieve the above purpose, the technical scheme of the invention is as follows: the humanoid robot control system for shoe and suit display is characterized by comprising a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC end software system; the power module comprises a power supply and a power supply electric quantity display device; the chassis control module comprises an industrial personal computer, a chassis control board, a laser radar, a lamp band controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves, an IR receiver, a motor driver and a motor; the network control module comprises a switch and a router; the visual voice module comprises a camera, a core board, a four-microphone base board, a microphone array, a power amplification board, a loudspeaker and a filter; the model motion control module comprises a controller, a stepping motor driver and a stepping motor; the chassis control board and the core board are in communication with the PC software system through the router.
The power supply provides electric energy for each module, power supply electric quantity display device passes through coulomb meter and chassis control panel connection communication, industrial computer and chassis control panel and laser radar are connection communication respectively, lamp area controller, crashproof switch, scram switch, ultrasonic wave, IR receiver and motor driver are connection communication with the chassis control panel respectively, motor driver is driven by it to the motor, the switch is with industrial computer, router and chassis control panel connection communication respectively, camera and core board are with router connection communication respectively, four wheat bottom plates and power amplifier board are with core board connection communication respectively, wave filter and loudspeaker are connected with the power amplifier board, the microphone display is connected with four wheat bottom plates, step motor driver is connected with the controller, step motor is connected with step motor driver, controller and chassis control panel connection communication.
The chassis control board adopts STM32 chassis control board and/or the core board adopts RK3288 core board.
The coulometer, the lamp band controller and the chassis control board are connected and communicated through an RS485 serial port; the industrial personal computer is connected and communicated with the laser radar and the switch through network port equipment; the router is connected and communicated with the core board and the PC end software system through network port equipment; the industrial personal computer is connected and communicated with the camera through a USB interface; the chassis control board is communicated with the anti-collision switch, the emergency stop switch and the controller through the GPIO port; the industrial personal computer is connected and communicated with the chassis control board through the CAN; the ultrasonic wave is communicated with the chassis control board through TTL; the motor driver is communicated with the chassis control board through CAN; the core board is connected and communicated with the four-wheat bottom board through an audio port and a USB interface; and/or; the core board is connected with the power amplifier board through an audio port for communication.
The operation mode of the humanoid robot control system comprises a navigation mode and a service mode, and the mode switching is controlled by a PC end software system; under a navigation mode, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of the model, and the visual voice module works to realize voice propaganda popularization of products; in a service mode, the visual voice module works to realize visual data acquisition and man-machine voice interaction communication; the PC side software system comprises a user UI interface and database management, wherein the main functions of the user UI interface comprise map building, navigation queuing, map and position information display, model motion control and service mode point configuration and display, and the main functions of the database management comprise visual identification information display, navigation points and service mode points.
The method for realizing man-machine voice interactive communication in the visual voice module is as follows,
1) After the user voice is output, the visual voice module acquires the user voice, and when the user voice is acquired, the input voice with higher quality is acquired through noise suppression, echo cancellation, sound source positioning and far-field pickup technologies;
2) The visual voice module analyzes the keywords of the user voice to obtain language keywords and/or related keywords of the user;
3) The method comprises the steps of carrying out interactive language processing on language keywords, specifically obtaining a knowledge base of a pre-established and stored natural language semantic template, carrying out interactive language processing to obtain an answer sentence content I, adopting a method of manually collecting, sorting and storing data and adopting a machine active learning method to accept user training, and collecting and organizing useful information to enrich the knowledge base;
User analysis is carried out on related keywords of the user, specifically, user analysis is carried out by obtaining information of a pre-established user library to obtain user information, the user information is added into the user library, and the user information is arranged to obtain second answer sentence content;
Meanwhile, a service system of the visual voice module performs product retrieval through a keyword analysis to obtain a result, specifically, obtains a product retrieval result through obtaining information of a pre-established product library and a user library, and obtains an answer sentence content III by arrangement;
4) The visual voice module synthesizes the first answer sentence content, the second answer sentence content and the third answer sentence content into a complete answer sentence;
5) Outputting the complete answer sentence.
The visual data acquisition in the visual voice module is used for estimating and analyzing the appearance and the body type of the user, specifically, the face recognition system and the body type recognition system in the visual voice module are used for estimating and analyzing to obtain user information comprising user face information, user gender information and user body type information, and the user information is added to a pre-established user library and/or is supplied to user analysis during human-computer voice interactive communication; the face recognition system comprises four steps, namely face image acquisition and face detection, face image preprocessing, face image feature extraction and face image matching and recognition.
The face detection in the face image acquisition and the face detection adopts an Adaboost learning algorithm; the face image feature extraction adopts a knowledge-based characterization method, and feature data which is beneficial to face classification is obtained according to the shape description of face organs and the distance characteristics between the face organs; and the face image matching and recognition is carried out by setting a threshold value, comparing the face features to be recognized with face feature templates obtained in a pre-established user library, outputting a matching result when the similarity exceeds the threshold value, and estimating gender information and age information of the recognized person according to the matching result.
The visual voice module adopts a convolutional neural network model to carry out gender prediction classification and age prediction classification, wherein gender prediction is used as a classification problem in the gender prediction classification, a face detected by a camera is used as input of a gender prediction network, a CNN (computer numerical network) is utilized to extract characteristics, an output layer of the gender prediction network is of a softmax type, and 2 nodes represent two categories of male and female as output; the age prediction problem is defined as a classification problem in the age prediction, the ages of 0-100 are divided into N age groups, and the age prediction network corresponds to N nodes at the last layer of softmax to represent the age range.
The body type recognition system divides the body type of a human body into five grades of a thin body type Y, a thinner body type YA, a common type A, a slightly fat type AB and a fat type B, extracts various image information characteristics according to standing characteristics of the human body, performs body type recognition by constructing a BP-Adaboost model, takes a BP neural network as a weak classifier, repeatedly trains a BP neural network prediction sample output, obtains a strong classifier composed of a plurality of BP neural network weak classifiers through an Adaboost algorithm, and finally performs body type grading and grading through threshold setting.
The calculating method is that after the image obtained by the camera is processed, the height information and the width information are extracted, the corresponding human body type grade is obtained through the following calculation, scoring, contrast and judgment,
Wherein H represents the body type height obtained after image processing, and W represents the body type width obtained after image processing;、、、 Representing different thresholds at different body type levels.
By adopting the technical scheme, the invention has the beneficial effects that: the invention discloses a humanoid robot control system, which comprises a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC end software system, wherein the PC end software system can realize various operation modes through the application of each equipment device of each module and the connection communication structure arrangement among each equipment period, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of a model, and the visual voice module works to realize voice propaganda popularization of products, human face recognition and body type recognition estimation analysis and man-machine voice interaction communication; the PC side software system comprises a user UI interface and database management, and can perform operations such as parameter adjustment, control, operation mode change and the like of the robot. In summary, the control system structure of the invention can enable the robot to achieve the high-automation and high-intelligent application of more humanized movement, interaction and propaganda popularization, and can be applied to big data collection, market research and product display popularization, and the application performance is good, thereby better realizing the above purpose effect of the invention.
Drawings
FIG. 1 is a block diagram of a humanoid robot control system for footwear-oriented display in accordance with the present invention;
FIG. 2 is a block diagram of a humanoid robot control system for footwear-oriented display in accordance with the present invention;
FIG. 3 is a block diagram of a model motion control module according to the present invention;
FIG. 4 is a block diagram of the operational mode of a humanoid robot control system presented for footwear according to the present invention;
FIG. 5 is a block diagram of a PC side software system according to the present invention;
FIG. 6 is a flow chart of man-machine voice interactive communication in a visual voice module according to the present invention;
fig. 7 is a schematic diagram of a convolutional neural network structure according to the present invention.
Detailed Description
In order to further explain the technical scheme of the invention, the invention is explained in detail by specific examples.
The embodiment discloses a humanoid robot control system for shoe and suit display, as shown in fig. 1, the system further comprises a power supply module 1, a chassis control module 2, a network control module 3, a model motion control module 4 and a visual voice module 5, and further comprises a PC end software system 6, wherein the power supply 1 provides electric energy for each module, the chassis control module 2 works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module 3 works to realize dynamic display of the model, the visual voice module 5 works to realize voice propaganda promotion of products and realize visual data acquisition to perform face recognition, body type recognition, estimation and analysis and man-machine voice interaction, and the structural layout and connection relation of equipment devices contained in each module are described in detail below in combination with fig. 2.
As shown in fig. 2, the power module 1 includes a power supply and a power supply amount display device; the chassis control module 2 comprises an industrial personal computer, a chassis control board (in the embodiment, STM32 chassis control board is adopted), a laser radar, a lamp belt controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves (4 are distributed), an IR receiver, a motor driver and a motor (in the embodiment, the number of the motor drivers and the number of the motors are respectively 2, and the robot chassis adopts independently suspended and driven travelling wheels and works together with universal wheels); the network control module 3 comprises a switch and a router; the visual voice module 5 comprises a camera, a core board (RK 3288 core board is adopted in the embodiment), a four-microphone base board, a microphone array, a power amplification board, speakers (2 speakers are arranged) and a filter, and the power supply is configured to adapt to the power supply according to the working condition; the model motion control module 4 includes a controller, a stepper motor driver, and a stepper motor.
The connection communication relationship of the equipment devices in this embodiment is specifically as follows: the chassis control board and the core board are in communication with the PC software system through the router. The utility model provides a power electric quantity display device, including the chassis control panel, the power supply electric quantity display device, the industrial computer passes through coulomb meter and chassis control panel connection communication, industrial computer and chassis control panel and laser radar are connection communication respectively, lamp area controller, crashproof switch, scram switch, ultrasonic wave, IR receiver and motor driver are connection communication with the chassis control panel respectively, motor driver is driven by it, the switch is with industrial computer, router and chassis control panel connection communication respectively, camera and core board are connection communication with the router respectively, four wheat bottom plates and power amplifier board are connection communication with the core board respectively, wave filter and loudspeaker are connected with the power amplifier board, the microphone is displayed and is connected with four wheat bottom plates, step motor driver is connected with the controller, step motor is connected with step motor driver, controller and chassis control panel connection communication. In the embodiment, the coulometer, the lamp band controller and the chassis control board are connected and communicated through an RS485 serial port; the industrial personal computer is connected and communicated with the laser radar and the switch through network port equipment; the router is connected and communicated with the core board and the PC end software system through network port equipment; the industrial personal computer is connected and communicated with the camera through a USB interface; the chassis control board is communicated with the anti-collision switch, the emergency stop switch and the controller through the GPIO port; the industrial personal computer is connected and communicated with the chassis control board through the CAN; the ultrasonic wave is communicated with the chassis control board through TTL; the motor driver is communicated with the chassis control board through CAN; the core board is connected and communicated with the four-wheat bottom board through an audio port and a USB interface; the core board is connected with the power amplifier board through an audio port for communication. The connection mode of the controller and the driver in the model motion control module 4 is shown as follows, wherein the PUL+ (pulse+), DIR+ (direction+) terminals of the stepping motor driver are connected with +5V, the PUL- (pulse-) terminal is connected with the PUL- (pulse-) of the controller, and the DIR- (direction-) terminal is connected with the DIR- (direction-) of the controller. The stepper motor driver power VCC (power positive) is connected to the power positive and GND (power negative) is connected to the power negative as shown in FIG. 3.
The control system structure can enable the robot to achieve the high-automation and high-intelligent application of more humanized movement, interaction and propaganda popularization.
The embodiment discloses an operation mode of the humanoid robot control system, which comprises a navigation mode and a service mode, as shown in fig. 4 and 5, the mode switching is controlled by a PC end software system 6, the robot can also control the movement and stop movement through the PC end software system, and the stop movement can also be controlled manually; under a navigation mode, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of the model, and the visual voice module works to realize voice propaganda popularization of products; in a service mode, the visual voice module works to realize visual data acquisition and man-machine voice interactive communication and provide directional guidance of popularization products according to the requirements of clients; the PC side software system comprises a user UI interface and database management, wherein the main functions of the user UI interface comprise drawing establishment, navigation queuing, map and position information display, model motion control and service mode point configuration and display, and the main functions of the database management comprise visual identification information display, navigation points and service mode points, namely, the functions of visual identification information storage, data classification, data display and the like. The PC end software system issues general instructions, communicates with the chassis control board through zmq, and communicates with the visual recognition system and the voice recognition system through TCP.
The autonomous navigation function is one of core technologies for displaying the operation of the humanoid robot in the indoor environment by the footwear, so that the environment information and the self state are perceived, and the autonomous movement of autonomous obstacle avoidance is realized. The information provided by the laser radar, the sensor and the like is integrated to form unified representation of an external environment, fusion of various information can play a complementary role, environmental characteristics are reflected after the instantaneity and redundancy of the information are guaranteed, so that correct judgment and decision are made, the robot is precisely positioned by utilizing the extended Kalman filtering algorithm to fuse laser and other sensor data, the data acquisition and processing of the environmental information are carried out through the carried laser radar to generate a local map, the map service is started to update in the global map, and the local map is circularly moved to be continuously scanned and updated in the environment until each update information of the local information is contained in the global map, and the map construction is completed.
Navigation is divided into global navigation and local navigation, and algorithms adopted in global navigation are an A-algorithm and a Di-Jie-Style algorithm, and the algorithm is responsible for track planning from a starting position to a target position. When the global path is determined, the shoe suit display humanoid robot adopts a track unfolding method or a dynamic window method, adopts local path planning in the real-time navigation process, namely local path planning, and is responsible for specific speed issuing and obstacle avoidance.
The method for realizing man-machine voice interactive communication in the visual voice module is that, as shown in figure 6,
1) After the user voice is output, the visual voice module acquires the user voice, and when the user voice is acquired, the input voice with higher quality is acquired through noise suppression, echo cancellation, sound source positioning and far-field pickup technologies;
2) The visual voice module analyzes the keywords of the user voice to obtain language keywords and/or related keywords of the user;
3) The method comprises the steps of carrying out interactive language processing on language keywords, specifically obtaining a knowledge base of a pre-established and stored natural language semantic template, carrying out interactive language processing to obtain an answer sentence content I, adopting a method of manually collecting, sorting and storing data and adopting a machine active learning method to accept user training, and collecting and organizing useful information to enrich the knowledge base;
User analysis is carried out on related keywords of the user, specifically, user analysis is carried out by obtaining information of a pre-established user library to obtain user information, the user information is added into the user library, and the user information is arranged to obtain second answer sentence content;
Meanwhile, a service system of the visual voice module performs product retrieval through a keyword analysis to obtain a result, specifically, obtains a product retrieval result through obtaining information of a pre-established product library and a user library, and obtains an answer sentence content III by arrangement;
4) The visual voice module synthesizes the first answer sentence content, the second answer sentence content and the third answer sentence content into a complete answer sentence;
5) Outputting the complete answer sentence.
The visual data acquisition in the visual voice module is used for estimating and analyzing the appearance and the body type of the user, specifically, the face recognition system and the body type recognition system in the visual voice module are used for estimating and analyzing to obtain user information comprising user face information, user gender information and user body type information, and the user information is added to a pre-established user library and/or is supplied to user analysis during human-computer voice interactive communication; the face recognition system comprises four steps, namely face image acquisition and face detection, face image preprocessing, face image feature extraction and face image matching and recognition. In summary, the visual recognition function is to collect the face information, sex information and shape information of the customer, so as to estimate the basic information of the customer and complete the recommendation of the corresponding footwear products according to the individual.
The face detection in the face image acquisition and face detection adopts an Adaboost learning algorithm, which is a method for classifying, wherein the algorithm selects some rectangular features (weak classifiers) which can most represent the face, the weak classifiers are constructed into a strong classifier according to a weighted voting mode, and a plurality of strong classifiers obtained through training are connected in series to form a cascade classifier with a cascade structure; the face image preprocessing is a process of processing an image based on a face detection result and finally serving for feature extraction, and the original image acquired by the system is subjected to image preprocessing such as gray correction, noise filtering and the like due to the limitation and random interference of various conditions; the face image feature extraction is also called face characterization, and is a process of modeling the face, and feature data which is helpful for face classification is obtained according to the shape description of face organs and the distance characteristics between the face organs by adopting a knowledge-based characterization method; and the face image matching and recognition is carried out by setting a threshold value, comparing the face features to be recognized with face feature templates obtained in a pre-established user library, outputting a matching result when the similarity exceeds the threshold value, and estimating gender information and age information of the recognized person according to the matching result.
The visual voice module adopts a convolutional neural network model to carry out gender prediction classification and age prediction classification, wherein gender prediction is used as a classification problem in the gender prediction classification, a face detected by a camera is used as input of a gender prediction network, a CNN (computer numerical network) is utilized to extract characteristics, an output layer of the gender prediction network is of a softmax type, and 2 nodes represent two categories of 'men' and 'women' as output; the age prediction problem is defined as a classification problem in the age prediction, and the ages of 0-100 are divided into N age groups, such as ages 0-2 are one group, ages 4-6 are another group, and so on. [ (0-2), (4-6), (8-12), (15-20), (25-32), (38-43), (48-53), (60-100) ], the age prediction network has N (8) nodes representing age ranges corresponding to the last layer of softmax. The embodiment of the invention discloses a method for carrying out gender prediction classification and age prediction classification by adopting a convolutional neural network model and parameter settings in the method, which can achieve a better functional effect of age prediction and body type prediction applied to a humanoid robot control system facing shoe suit display.
Firstly, the input process scales a given image to an M-by-M proportion, then 3-channel color image processing is carried out, if the image does not accord with the size, redundant frames are needed to be cut, and the cutting process is to cut from the center to four sides of the picture. The specific convolutional neural network processing is performed, and the invention is described with respect to using 3 convolutional layers and2 fully-connected layers, as shown in fig. 7.
First convolution layer: 96 convolution kernels are adopted, the number of parameters of each convolution kernel is 3 x 7, the activation function adopts ReLU, pooling adopts maximum overlapping pooling, pooled size is 3 x 3, stride is 2, and convolution step length is 4. The layer is then normalized for the local response.
Wherein the method comprises the steps ofThe output result after the convolution layer is represented, namely, a four-dimensional number (batch, height, width, channel), batch represents batch times, height represents picture height, width represents picture width, and channel is channel number.A position a, b, c, d in this output structure is shown, i.e. the point of height b and width c under the d-th channel of the a-th diagram.
Second convolution layer: the input of the second layer is used for processing the processed single-channel picture again, 256 filters are selected, the size of each filter is 5^2, and the convolution step length is 1.
Third convolution layer: the number of filters is selected 384, and the convolution kernel size is 3^2.
For the fully connected layer: the number of neurons per layer is selected 2^9.
Output layer: for gender, two classifications, input neuron number is 2; for age, 8 categories, the number of input neurons is 8.
Training process:
(1) Initializing parameters: the weight initialization method adopts Gao Sizheng too distribution with standard deviation of 0.01 and average value of 0.
(2) Training a network: dropout is used to limit the overfit. The Drop out ratio is 0.5, and the data expansion is realized by inputting a picture of M x M and then clipping.
(3) The training method adopts a random gradient descent method, the min-batch size is selected to be 50, the learning rate is 0.001, and then the learning rate is adjusted to be 0.0001 after the iteration is performed for 10000 times.
(4) And (3) predicting results: the prediction method adopts a picture of 256 x 256, then processes for multiple times, and finally averages the prediction result.
The body type recognition system divides the body type of a human body into five grades of a thin body type Y, a thinner body type YA, a common type A, a slightly fat type AB and a fat type B, extracts various image information characteristics according to standing characteristics of the human body, performs body type recognition by constructing a BP-Adaboost model, takes a BP neural network as a weak classifier, repeatedly trains a BP neural network prediction sample output, obtains a strong classifier composed of a plurality of BP neural network weak classifiers through an Adaboost algorithm, and finally performs body type grading and grading through threshold setting.
The calculating method is that after the image obtained by the camera is processed, the height information and the width information are extracted, the corresponding human body type grade is obtained through the following calculation, scoring, contrast and judgment,
Wherein H represents the body type height obtained after image processing, and W represents the body type width obtained after image processing;、、、 Representing different thresholds at different body type levels.
The above examples and drawings are not intended to limit the form or form of the present invention, and any suitable variations or modifications thereof by those skilled in the art should be construed as not departing from the scope of the present invention.
Claims (6)
1. The humanoid robot control system for shoe and suit display is characterized by comprising a power supply module, a chassis control module, a network control module, a model motion control module, a visual voice module and a PC end software system; the power module comprises a power supply and a power supply electric quantity display device; the chassis control module comprises an industrial personal computer, a chassis control board, a laser radar, a lamp band controller, a coulometer, an anti-collision switch, an emergency stop switch, ultrasonic waves, an IR receiver, a motor driver and a motor; the network control module comprises a switch and a router; the visual voice module comprises a camera, a core board, a four-microphone base board, a microphone array, a power amplification board, a loudspeaker and a filter; the model motion control module comprises a controller, a stepping motor driver and a stepping motor; the chassis control board and the core board are in communication with a PC end software system through a router;
The power supply provides electric energy for each module, the power supply electric quantity display device is in connection communication with a chassis control board through a coulometer, the industrial personal computer is in connection communication with the chassis control board and the laser radar respectively, the lamp belt controller, the anti-collision switch, the emergency stop switch, the ultrasonic wave, the IR receiver and the motor driver are in connection communication with the chassis control board respectively, the motor driver is connected with the motor and driven by the motor, the switch is in connection communication with the industrial personal computer, the router and the chassis control board respectively, the camera and the core board are in connection communication with the router respectively, the four-microphone bottom board and the power amplification board are in connection communication with the core board respectively, the filter and the loudspeaker are connected with the power amplification board, the microphone display is connected with the four-microphone bottom board, the stepper motor driver is connected with the controller, the stepper motor is connected with the stepper motor driver, and the controller is in connection communication with the chassis control board;
The chassis control board adopts STM32 chassis control board and/or the core board adopts RK3288 core board; the coulometer, the lamp band controller and the chassis control board are connected and communicated through an RS485 serial port; the industrial personal computer is connected and communicated with the laser radar and the switch through network port equipment; the router is connected and communicated with the core board and the PC end software system through network port equipment; the industrial personal computer is connected and communicated with the camera through a USB interface; the chassis control board is communicated with the anti-collision switch, the emergency stop switch and the controller through the GPIO port; the industrial personal computer is connected and communicated with the chassis control board through the CAN; the ultrasonic wave is communicated with the chassis control board through TTL; the motor driver is communicated with the chassis control board through CAN; the core board is connected and communicated with the four-wheat bottom board through an audio port and a USB interface; and/or; the core board is communicated with the power amplification board through an audio port;
The operation mode of the humanoid robot control system comprises a navigation mode and a service mode, and the mode switching is controlled by a PC end software system; under a navigation mode, the chassis control module works to realize autonomous navigation, path planning and obstacle avoidance of the humanoid robot, the model motion control module works to realize dynamic display of the model, and the visual voice module works to realize voice propaganda popularization of products; in a service mode, the visual voice module works to realize visual data acquisition and man-machine voice interaction communication; the PC side software system comprises a user UI interface and database management, wherein the main functions of the user UI interface comprise map building, navigation queuing, map and position information display, model motion control and service mode point configuration and display, and the main functions of the database management comprise visual identification information display, navigation points and service mode points;
The visual data acquisition in the visual voice module is used for estimating and analyzing the appearance and the body type of the user, specifically, the face recognition system and the body type recognition system in the visual voice module are used for estimating and analyzing to obtain user information comprising user face information, user gender information and user body type information, and the user information is added to a pre-established user library and/or is supplied to user analysis during human-computer voice interactive communication; the face recognition system comprises four steps, namely face image acquisition, face detection, face image preprocessing, face image feature extraction and face image matching and recognition in sequence;
The face detection in the face image acquisition and the face detection adopts an Adaboost learning algorithm; the face image feature extraction adopts a knowledge-based characterization method, and feature data which is beneficial to face classification is obtained according to the shape description of face organs and the distance characteristics between the face organs; and the face image matching and recognition is carried out by setting a threshold value, comparing the face features to be recognized with face feature templates obtained in a pre-established user library, outputting a matching result when the similarity exceeds the threshold value, and estimating gender information and age information of the recognized person according to the matching result.
2. The system for controlling a humanoid robot on a footwear-oriented display according to claim 1, wherein the method for implementing man-machine voice interactive communication in the visual voice module is such that,
1) After the user voice is output, the visual voice module acquires the user voice, and when the user voice is acquired, the input voice with higher quality is acquired through noise suppression, echo cancellation, sound source positioning and far-field pickup technologies;
2) The visual voice module analyzes the keywords of the user voice to obtain language keywords and/or related keywords of the user;
3) The method comprises the steps of carrying out interactive language processing on language keywords, specifically obtaining a knowledge base of a pre-established and stored natural language semantic template, carrying out interactive language processing to obtain an answer sentence content I, adopting a method of manually collecting, sorting and storing data and adopting a machine active learning method to accept user training, and collecting and organizing useful information to enrich the knowledge base;
User analysis is carried out on related keywords of the user, specifically, user analysis is carried out by obtaining information of a pre-established user library to obtain user information, the user information is added into the user library, and the user information is arranged to obtain second answer sentence content;
Meanwhile, a service system of the visual voice module performs product retrieval through a keyword analysis to obtain a result, specifically, obtains a product retrieval result through obtaining information of a pre-established product library and a user library, and obtains an answer sentence content III by arrangement;
4) The visual voice module synthesizes the first answer sentence content, the second answer sentence content and the third answer sentence content into a complete answer sentence;
5) Outputting the complete answer sentence.
3. The humanoid robot control system for shoe suit presentation according to claim 1, wherein a convolutional neural network model is adopted in the visual voice module to perform gender prediction classification and age prediction classification, wherein gender prediction is used as a classification problem in the gender prediction classification, a face detected by a camera is used as an input of a gender prediction network, a CNN (computer numerical network) is utilized to extract characteristics, an output layer of the gender prediction network is of a softmax type, and 2 nodes represent two categories of male and female as outputs; the age prediction problem is defined as a classification problem in the age prediction, the ages of 0-100 are divided into N age groups, and the age prediction network corresponds to N nodes at the last layer of softmax to represent the age range.
4. The humanoid robot control system for shoe-wear display according to claim 1, wherein the body type recognition system classifies the body type into five grades of thin body type Y, thin body type YA, common type A, slightly fat type AB and fat type B, extracts various image information features according to the standing features of the body, performs body type recognition by constructing a BP-Adaboost model, takes a BP neural network as a weak classifier, repeatedly trains BP neural network prediction samples, outputs the BP neural network prediction samples, obtains a strong classifier composed of a plurality of BP neural network weak classifiers through an Adaboost algorithm, and finally performs body type scoring and classification through threshold setting.
5. The shoe-suit-display-oriented humanoid robot control system of claim 4, wherein the calculation method is that after the image processing obtained by a camera, the height information and the body width information are extracted, the corresponding human body type grade is obtained through the following calculation, scoring, comparison and judgment,Wherein H represents the body type height obtained after image processing, and W represents the body type width obtained after image processing;、、、 Representing different thresholds at different body type levels.
6. A footwear-oriented presentation humanoid robot control system as claimed in claim 3, characterised in that the age of 0-100 is divided into 8 age groups; in gender prediction classification and age prediction classification by adopting a convolutional neural network model, 3 convolutional layers and 2 full-connection layers are adopted, firstly, a given image is scaled to M-to-M proportion in an input process, then 3-channel color image processing is carried out, if the image does not accord with the size, redundant frames are required to be cut, and the cutting process is cutting from the center to four sides of the picture; first convolution layer: 96 convolution kernels are adopted, the number of parameters of each convolution kernel is 3 x 7, an activation function adopts a ReLU, pooling adopts maximum overlapping pooling, pooled size is 3 x 3, stride is 2, convolution step length is 4, and then a local response normalization layer is carried out; second convolution layer: the input of the second layer is used for processing the processed single-channel picture again, 256 filters are selected, the size of each filter is 5^2, and the convolution step length is 1; third convolution layer: the number of the filters is 384, and the convolution kernel size is 3^2; for the fully connected layer: the number of neurons in each layer is selected 2^9; output layer: for gender prediction, two classifications, input neuron number 2; for age prediction, the number of input neurons is 8, which is a class of 8; in the model training process, firstly, initializing parameters, wherein the weight initialization method adopts Gao Sizheng too distribution with standard deviation of 0.01 and mean value of 0; then, performing network training, adopting Drop out to limit overfitting, adopting Drop out proportion to be 0.5, adopting data expansion by inputting pictures of M.M, then performing clipping, adopting a random gradient descent method for training, selecting a min-batch size to be 50, adopting a learning rate to be 0.001, and then adjusting the learning rate to be 0.0001 after iteration is performed for 10000 times; finally, the result is predicted, and the prediction method adopts a picture of 256 to be input, and carries out multiple processing to average the prediction result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536802.6A CN112578716B (en) | 2020-12-23 | 2020-12-23 | Humanoid robot control system for shoe-suit display |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536802.6A CN112578716B (en) | 2020-12-23 | 2020-12-23 | Humanoid robot control system for shoe-suit display |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112578716A CN112578716A (en) | 2021-03-30 |
| CN112578716B true CN112578716B (en) | 2024-07-02 |
Family
ID=75139431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011536802.6A Active CN112578716B (en) | 2020-12-23 | 2020-12-23 | Humanoid robot control system for shoe-suit display |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112578716B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN213843825U (en) * | 2020-12-23 | 2021-07-30 | 福建（泉州）哈工大工程技术研究院 | Humanoid robot control system for shoe and clothes display |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201105467A (en) * | 2009-08-04 | 2011-02-16 | Mirle Automation Corp | Robotic fashion model |
| CN106346487B (en) * | 2016-08-25 | 2018-09-21 | 威仔软件科技（苏州）有限公司 | Interactive VR sand table show robot |
| CN206085055U (en) * | 2016-09-28 | 2017-04-12 | 深圳维周机器人科技有限公司 | Intelligence control system of model robot |
| CN110403409A (en) * | 2019-07-18 | 2019-11-05 | 王东 | A kind of robot and merchandise display method |
| CN211137173U (en) * | 2019-12-19 | 2020-07-31 | 大连雨林信息***有限公司 | Robot for clothing display |
-
2020
- 2020-12-23 CN CN202011536802.6A patent/CN112578716B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN213843825U (en) * | 2020-12-23 | 2021-07-30 | 福建（泉州）哈工大工程技术研究院 | Humanoid robot control system for shoe and clothes display |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112578716A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1406135B1 (en) | Man-machine interface unit control method; robot apparatus; and its action control method | |
| CN110175596B (en) | Virtual learning environment micro-expression recognition and interaction method based on double-current convolutional neural network | |
| CN108780523B (en) | Cloud-based processing using sensor data and tags provided by local devices | |
| US11417148B2 (en) | Human face image classification method and apparatus, and server | |
| CN107766787B (en) | Face attribute identification method, device, terminal and storage medium | |
| US7624076B2 (en) | Weak hypothesis generation apparatus and method, learning apparatus and method, detection apparatus and method, facial expression learning apparatus and method, facial expression recognition apparatus and method, and robot apparatus | |
| CN111496802A (en) | Control method, device, equipment and medium for artificial intelligence equipment | |
| Chen et al. | Adaptive feature selection-based AdaBoost-KNN with direct optimization for dynamic emotion recognition in human–robot interaction | |
| Umamaheswari et al. | An enhanced human speech emotion recognition using hybrid of PRNN and KNN | |
| CN102254180B (en) | Geometrical feature-based human face aesthetics analyzing method | |
| CN111199202B (en) | Human body action recognition method and recognition device based on circulating attention network | |
| CN109815920A (en) | Gesture identification method based on convolutional neural networks and confrontation convolutional neural networks | |
| CN113343860A (en) | Bimodal fusion emotion recognition method based on video image and voice | |
| KR20200126675A (en) | Electronic device and Method for controlling the electronic device thereof | |
| CN213843825U (en) | Humanoid robot control system for shoe and clothes display | |
| Tang et al. | Facial expression recognition using AAM and local facial features | |
| Brandizzi et al. | Automatic rgb inference based on facial emotion recognition | |
| CN112578716B (en) | Humanoid robot control system for shoe-suit display | |
| Wu et al. | From members to teams to committee-a robust approach to gestural and multimodal recognition | |
| Wenzheng | Human activity recognition based on acceleration sensor and neural network | |
| CN112069916A (en) | Face beauty prediction method, device and system and readable storage medium | |
| Austermann et al. | Prosody based emotion recognition for MEXI | |
| CN110363074A (en) | One kind identifying exchange method for complicated abstract class of things peopleization | |
| Chen et al. | Multimodal emotion recognition and intention understanding in human-robot interaction | |
| CN113537335B (en) | Method and system for analyzing hand assembly skills |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: 362000 Building 9, software park, Fengze District, Quanzhou City, Fujian Province Applicant after: Fujian (Quanzhou) Advanced Manufacturing Technology Research Institute Address before: 362000 Building 9, software park, Fengze District, Quanzhou City, Fujian Province Applicant before: FUJIAN (QUANZHOU) HIT Research Institute OF ENGINEERING AND TECHNOLOGY Country or region before: China |
|
| GR01 | Patent grant |