CN110276302A - The method and system of elevator are taken by a kind of robot - Google Patents
The method and system of elevator are taken by a kind of robot Download PDFInfo
- Publication number
- CN110276302A CN110276302A CN201910551001.8A CN201910551001A CN110276302A CN 110276302 A CN110276302 A CN 110276302A CN 201910551001 A CN201910551001 A CN 201910551001A CN 110276302 A CN110276302 A CN 110276302A
- Authority
- CN
- China
- Prior art keywords
- elevator
- feature
- predeterminable area
- testing result
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0006—Monitoring devices or performance analysers
- B66B5/0012—Devices monitoring the users of the elevator system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to robot field, the method and system that elevator is taken by a kind of robot are disclosed, method includes: to obtain the image of elevator interior when elevator door is opened；Described image is inputted in trained neural network model in advance, object detection results and predeterminable area testing result are obtained, the predeterminable area is the region that robot enters that elevator needs to occupy；According to the object detection results and predeterminable area testing result, judge whether that the elevator can be entered, if so, if it is not, then discharging the elevator, and calling elevator again into the elevator.The present invention detects the object whether predeterminable area in elevator is occupied and occupies by obtaining elevator interior image, to determine whether can enter elevator, when elevator can not be entered, elevator is directly discharged, the time for excessively occupying passenger-in-elevator is avoided, to improve the friendly interaction capabilities of robot.
Description
Technical field
The invention belongs to robotic technology field, in particular to the method and system of elevator are taken by a kind of robot.
Background technique
In recent years, as the development of robot technology and artificial intelligence study deepen continuously, intelligent mobile robot is in people
Play the part of more and more important role in class life, is used widely in numerous areas.In certain applications, robot may
It can be used in the scene across floor, for example send article across floor.In this scene, robot generally requires to take elevator in fact
Now across floor operation.
And lift space is narrow, robot enter elevator when, if remaining space be not enough to accommodate robot or
The travel path of person robot is blocked, then robot will have to exit elevator.When such a situation occurs, robot
It into elevator and drops by the wayside and will consume the time of passenger-in-elevator, so that robot lacks friendly interaction capabilities to a certain extent.
Summary of the invention
The object of the present invention is to provide the method and system that elevator is taken by a kind of robot, can avoid waste passenger-in-elevator when
Between, improve the friendly interaction capabilities of robot.
Technical solution provided by the invention is as follows:
On the one hand, a kind of method that elevator is taken by robot is provided, comprising:
When elevator door is opened, the image of elevator interior is obtained；
Described image is inputted in trained neural network model in advance, object detection results and predeterminable area inspection are obtained
It surveys as a result, the predeterminable area is the region that robot enters that elevator needs to occupy；
According to the object detection results and predeterminable area testing result, judge whether that the elevator can be entered, if so,
Into the elevator, if it is not, then discharging the elevator, and elevator is called again.
It is further preferred that described according to the object detection results and predeterminable area testing result, judging whether can be into
Enter the elevator to specifically include:
According to the predeterminable area testing result, judge whether the predeterminable area is occupied, if it is not, then determining to enter
The elevator；
If judging that the predeterminable area is occupied, then according to the target detection according to the predeterminable area testing result
As a result, judgement occupies whether object is object；
If the object that occupies is object, judgement can not enter the elevator；
If the object that occupies is not object, evacuation information is exported, after waiting preset time, is obtained in the elevator again
The new images in portion input the new images in the neural network model, obtain object detection results and predeterminable area again
Testing result, and according to the object detection results and predeterminable area testing result obtained again, judge whether that the electricity can be entered
Ladder when repeating to obtain the number of image of the elevator interior greater than preset threshold then determines that the elevator can not be entered.
It is further preferred that described input described image in trained neural network model in advance, target inspection is obtained
It surveys result and predeterminable area testing result specifically includes:
Described image is inputted in trained neural network model in advance, the characteristic pattern of different layers is extracted；
Fusion Features and increase resolution are carried out to the characteristic pattern of different layers；
According to the characteristic pattern obtained after Fusion Features and increase resolution, object detection results and predeterminable area detection are exported
As a result.
It is further preferred that described input described image in trained neural network model in advance, difference is extracted
The characteristic pattern of layer specifically includes:
Described image is inputted in trained neural network model in advance, the fisrt feature figure of the last layer is extracted,
And the second feature figure of front layer is extracted, the resolution ratio of the second feature figure is the two of the resolution ratio of the fisrt feature figure
Times；
The characteristic pattern to different layers carries out Fusion Features and increase resolution specifically includes:
According to default step, the fisrt feature figure is handled, makes the increase resolution one of the fisrt feature figure
Times, obtain third feature figure；
It will be merged after the second feature figure 1 × 1 process of convolution of progress with the third feature figure, obtain the 4th spy
Sign figure；
According to the default step, the fourth feature figure is repeatedly handled, by the resolution of the fourth feature figure
Rate promotes more times, obtains fifth feature figure；
It is described according to the characteristic pattern obtained after Fusion Features and increase resolution, export object detection results and predeterminable area
Testing result specifically includes:
According to the fifth feature figure, object detection results and predeterminable area testing result are exported.
It is further preferred that it is described according to default step, the fisrt feature figure is handled, the fisrt feature is made
One times of the increase resolution of figure obtains third feature figure and specifically includes:
Deconvolution is carried out to the fisrt feature figure；
Channel separation is carried out to the fisrt feature figure after deconvolution；
Process of convolution is carried out to the fisrt feature figure after channel separation using two different convolution kernels；
Channel fusion is carried out to the characteristic pattern obtained after process of convolution, obtains third feature figure.
On the other hand, a kind of system that elevator is taken by robot is also provided, comprising:
Image collection module, for obtaining the image of elevator interior when elevator door is opened；
Detection module obtains target detection knot for inputting described image in trained neural network model in advance
Fruit and predeterminable area testing result, the predeterminable area are the region that robot enters that elevator needs to occupy；
Processing module, for judging whether that institute can be entered according to the object detection results and predeterminable area testing result
Elevator is stated, if so, if it is not, then discharging the elevator, and calling elevator again into the elevator.
It is further preferred that the processing module includes:
Processing unit, for judging whether the predeterminable area is occupied according to the predeterminable area testing result, if
It is no, then determine that the elevator can be entered；
The processing unit, if being also used to judge that the predeterminable area is occupied according to the predeterminable area testing result,
Then according to the object detection results, judgement occupies whether object is object；
The processing unit, if being also used to the object that occupies is object, judgement can not enter the elevator；
The processing unit exports evacuation information if being also used to the object that occupies is not object, waits preset time
Afterwards, the new images are inputted in the neural network model, obtain mesh again by the new images for obtaining the elevator interior again
Mark testing result and predeterminable area testing result, and according to the object detection results and predeterminable area testing result obtained again,
Judge whether that the elevator can be entered, when repeating to obtain the number of image of the elevator interior greater than preset threshold, then sentences
Surely it can not enter the elevator.
It is further preferred that the detection module includes:
Feature extraction unit extracts difference for inputting described image in trained neural network model in advance
The characteristic pattern of layer；
Increase resolution unit carries out Fusion Features and increase resolution for the characteristic pattern to different layers；
Task output unit, for exporting target detection according to the characteristic pattern obtained after Fusion Features and increase resolution
As a result with predeterminable area testing result.
It is further preferred that the feature extraction unit, is also used to inputting described image into trained nerve net in advance
In network model, the fisrt feature figure of the last layer is extracted, and extracts the second feature figure of front layer, the second feature figure
Resolution ratio be twice of resolution ratio of the fisrt feature figure；
The increase resolution unit includes:
Increase resolution subelement, for handling the fisrt feature figure, making described first according to default step
One times of the increase resolution of characteristic pattern, obtains third feature figure；
Fusion Features subelement, for by the second feature figure carry out 1 × 1 process of convolution after with the third feature figure
It is merged, obtains fourth feature figure；
The increase resolution subelement, for repeatedly being located to the fourth feature figure according to the default step
Reason, by more times of increase resolution of the fourth feature figure, obtains fifth feature figure；
The task output unit is also used to export object detection results and predeterminable area according to the fifth feature figure
Testing result.
It is further preferred that the increase resolution subelement, is also used to carry out deconvolution to the fisrt feature figure；It is right
The fisrt feature figure after deconvolution carries out channel separation；Using two different convolution kernels to described after channel separation
One characteristic pattern carries out process of convolution；Channel fusion is carried out to the characteristic pattern obtained after process of convolution, obtains third feature figure.
Compared with prior art, the method and system that elevator is taken by a kind of robot provided by the invention have beneficial below
Effect: the present invention detects pair whether predeterminable area in elevator is occupied and occupies by obtaining elevator interior image
As, to determine whether elevator can be entered, when elevator can not be entered, elevator is directly discharged, avoids the time for excessively occupying passenger-in-elevator,
To improve the friendly interaction capabilities of robot；In addition, the present invention by obtain the detection mode of image and neural network model come
Judge whether that elevator can be entered, avoiding can not examine using causing to be blocked because not knowing object category when depth transducer partially
It measures, so that the problem of space judges incorrectly, improves judging nicety rate.
Detailed description of the invention
Below by clearly understandable mode, preferred embodiment is described with reference to the drawings, elevator is taken to a kind of robot
Above-mentioned characteristic, technical characteristic, advantage and its implementation of method and system be further described.
Fig. 1 is the flow diagram of the first embodiment for the method that elevator is taken by a kind of robot of the present invention；
Fig. 2 is the flow diagram of the second embodiment for the method that elevator is taken by a kind of robot of the present invention；
Fig. 3 is the flow diagram of the 3rd embodiment for the method that elevator is taken by a kind of robot of the present invention；
Fig. 4 is the flow diagram of the fourth embodiment for the method that elevator is taken by a kind of robot of the present invention；
Fig. 5 is the flow diagram for the example that the present invention carries out Fusion Features and increase resolution to characteristic pattern；
Fig. 6 is the structural schematic block diagram of the one embodiment for the system that elevator is taken by a kind of robot of the present invention.
Drawing reference numeral explanation
10, image collection module；20, detection module；30, processing module.
Specific embodiment
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, Detailed description of the invention will be compareed below
A specific embodiment of the invention.It should be evident that drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing, and obtain other embodiments.
It should be appreciated that when used in this manual, term " includes " indicates the Expressive Features, entirety, step, behaviour
Make, the presence of element and/or component, but one or more other features, entirety, step, operation, element, component are not precluded
And/or the presence or addition of set.
To make simplified form, part related to the present invention is only schematically shown in each figure, they are not represented
Its practical structures as product.In addition, there is identical structure or function in some figures so that simplified form is easy to understand
Component only symbolically depicts one of those, or has only marked one of those.Herein, "one" is not only indicated
" only this ", can also indicate the situation of " more than one ".
The first embodiment provided according to the present invention, as shown in Figure 1, a kind of method that elevator is taken by robot, comprising:
S100 obtains the image of elevator interior when elevator door is opened；
S200 inputs described image in trained neural network model in advance, obtains object detection results and preset areas
Domain testing result, the predeterminable area are the region that robot enters that elevator needs to occupy；
S300 according to the object detection results and predeterminable area testing result, judge whether can enter the elevator, if
It is then to enter the elevator, if it is not, then discharging the elevator, and call elevator again.
Specifically, when robot needs to take elevator, robot first runs to elevator position, then calling electricity
Ladder；When elevator door is opened, robot face elevator obtains the figure of elevator interior by the camera being mounted in robot
Picture.Camera in addition to mountable in robot, it is also mountable in elevator, when robot call elevator after, server scheduling
Destination where target elevator to robot, when the destination where target elevator arrival robot, server control
Camera in elevator processed obtains the image of elevator interior.
After getting the image of elevator interior, by preparatory trained neural network model, the image of acquisition is carried out
Identification obtains object detection results and predeterminable area testing result.Target refers to relatively common object or person, for example, curing
When in institute's scene, target refers to people or in position classifications objects such as the relatively common hospital bed of hospital, cart, wheelchairs.Predeterminable area
Refer to that robot is able to enter the position occupied required for elevator.Due to the size of robot be it is known, can be in elevator
Middle extract sets the predeterminable area that can accommodate robot, and the size of predeterminable area is greater than the size of robot.Robot exists
When into elevator, only moves forward or back, can not also be moved left and right, therefore, predeterminable area is a fixation in elevator
Region.
Neural network model is improved with Objects as Points foundation structure, is used in basic network part
The structure of mobilenetv3 extracts feature, and joined intermediate module, intermediate module be used for from
The characteristic pattern extracted in mobilenetv3 carries out increase resolution, and the characteristic pattern after increase resolution is then inputted more
Output module be engaged in output test result, i.e., is added to the branch of region decision on the basis of Objects asPoints.
After neural network model is built, the image of a large amount of elevator interior is obtained, then by way of manually marking,
Target is marked out in the picture and state that predeterminable area is occupied；Collection and test set is respectively trained in the image marked, is led to
It crosses in the neural network model that training set builds input and is trained, then by test set to trained neural network
Model is tested, and according to test result, carries out data enhancing to the specific training sample in training set, then will be enhanced
Data set is further divided into training set and test set, and trained neural network model is trained and is tested again, repeated data
The step of collection enhancing, training and test, until obtaining a limit detection effect, obtain trained neural network model.
After obtaining object detection results and predeterminable area testing result, it can judge whether enter according to the result of acquisition
Elevator.For example, if detecting, predeterminable area is not occupied, illustrates that robot can be directly entered elevator, if detecting preset areas
Domain is occupied by hospital bed, then illustrates that robot can not enter elevator, need to discharge the elevator, call other elevators again.
In the present invention, by obtaining elevator interior image, and detect whether the predeterminable area in elevator is occupied and accounts for
According to object, to determine whether can enter elevator, when elevator can not be entered, directly release elevator, avoid excessively occupying passenger-in-elevator
Time, to improve the friendly interaction capabilities of robot；In addition, detection of the present invention by acquisition image and neural network model
Mode is to determine whether can enter elevator, avoiding leads to the part that is blocked because not knowing object category using when depth transducer
It can not detect, so that the problem of space judges incorrectly, improves judging nicety rate.
The second embodiment provided according to the present invention, as shown in Fig. 2, a kind of method that elevator is taken by robot, above-mentioned
On the basis of first embodiment, according to the object detection results and predeterminable area testing result, judge whether to enter described
Elevator specifically includes:
According to the predeterminable area testing result, judge whether the predeterminable area is occupied, if it is not, then determining to enter
The elevator；
If judging that the predeterminable area is occupied, then according to the target detection according to the predeterminable area testing result
As a result, judgement occupies whether object is object；
If the object that occupies is object, judgement can not enter the elevator；
If the object that occupies is not object, evacuation information is exported, after waiting preset time, is obtained in the elevator again
The new images in portion input the new images in the neural network model, obtain object detection results and predeterminable area again
Testing result, and according to the object detection results and predeterminable area testing result obtained again, judge whether that the electricity can be entered
Ladder when repeating to obtain the number of image of the elevator interior greater than preset threshold then determines that the elevator can not be entered.
Specifically, after obtaining object detection results and predeterminable area testing result, first according to predeterminable area testing result,
Judge the state of predeterminable area, if predeterminable area is not occupied, can directly judge that robot can enter elevator.
If at least 10% (can be set according to the actual situation in discovery predeterminable area according to predeterminable area testing result
Set) region when being classified as occupy state, the object of predeterminable area is occupied according to object detection results analysis.If occupying default
Region be the non-pedestrian such as hospital bed when, then determine that elevator, directly release elevator can not be entered, and call other elevators again, into
Enter new detection and judges process.
If that occupy predeterminable area is pedestrian, robot carries out voice broadcast, prompts pedestrian's evacuation；Then it waits default
After time (as waited 5 seconds), then the new images for obtaining the elevator interior are re-shoot, and by neural network model to new images
It is detected, new object detection results and predeterminable area testing result is obtained, to judge whether pedestrian avoids.If row
People has avoided, then judges that robot can enter elevator, robot drives directly into elevator.
If pedestrian goes back and avoids, voice broadcast is carried out again, prompts pedestrian's evacuation, after waiting preset time, then
Secondary progress entirely judges process, that is, re-executes above-mentioned steps S100, S200 and S300；Process is entirely judged when repeating
When number is more than preset threshold (such as 2 times, 3 inferior), then illustrate the pedestrian in elevator can not avoid or elevator in pedestrian
It is unwilling to be avoided, at this point, directly discharging elevator, abandons this boarding, and call elevator again.
In the present solution, by repeated detection and judgement, whether distinguishable pedestrian out is friendly when predeterminable area is occupied by pedestrian
Good interaction directly discharges elevator when pedestrian is unfriendly interactive, avoids the time of waste pedestrian and robot itself.
The 3rd embodiment provided according to the present invention, as shown in figure 3, the method that elevator is taken by a robot, above-mentioned
On the basis of the first embodiment or the second embodiment, described image is inputted trained neural network model in advance by step S200
In, it obtains object detection results and predeterminable area testing result specifically includes:
S210 inputs described image in trained neural network model in advance, extracts the characteristic pattern of different layers；
S220 carries out Fusion Features and increase resolution to the characteristic pattern of different layers；
S230 exports object detection results and predeterminable area according to the characteristic pattern obtained after Fusion Features and increase resolution
Testing result.
Specifically, after the image for the elevator interior that will acquire inputs in trained neural network model, pass through
Mobilenetv3 extracts the characteristic pattern of different layers.In convolutional neural networks, high-level characteristic pattern has stronger semanteme
Property, lower resolution ratio maps out global and profile feature.The characteristic pattern of low level has weaker Semantic, higher
Resolution ratio maps out part and minutia.
Fusion Features are carried out to the characteristic patterns of different layers, feature in characteristic pattern to combine low level and high-level
Feature in characteristic pattern, and then Mutually fusion is carried out to feature, resolution ratio then is carried out to the characteristic pattern obtained after Fusion Features
It is promoted, further according to the characteristic pattern obtained after increase resolution, obtains object detection results and predeterminable area testing result.By right
Feature carries out fusion and increase resolution, can take into account well and carry out detection to the target in image and to the region in image
It is detected.
Illustratively, the acquisition process of predeterminable area testing result are as follows: assuming that carrying out the characteristic pattern after increase resolution
Resolution ratio is the 1/n of the image of the elevator interior (original image) obtained, that is, each pair of point in characteristic pattern finally obtained answers original image
In a n × n region, in the characteristic pattern finally obtained each point carry out classification and Detection, i.e., to each n in original image
The region of × n carries out classification and Detection, judges that whether each region is occupied in original image, then therefrom extracts in predeterminable area
Whether each region n × n is occupied, and predeterminable area testing result can be obtained.
In the present solution, being carried out by the feature in the characteristic pattern to low level and the feature in high-level characteristic pattern mutual
Fusion is mended, the high efficiency and accuracy rate to target detection can be improved.
The fourth embodiment provided according to the present invention, as shown in figure 4, the method that elevator is taken by a robot, comprising:
S100 obtains the image of elevator interior when elevator door is opened；
S211 inputs described image in trained neural network model in advance, extracts the fisrt feature of the last layer
Figure, and the second feature figure of front layer is extracted, the resolution ratio of the second feature figure is the resolution ratio of the fisrt feature figure
Twice；
S221 is handled the fisrt feature figure according to default step, proposes the resolution ratio of the fisrt feature figure
One times is risen, third feature figure is obtained；
S222 will the second feature figure carry out 1 × 1 process of convolution after merge with the third feature figure, obtain the
Four characteristic patterns；
S223 is repeatedly handled the fourth feature figure according to the default step, by the fourth feature figure
More times of increase resolution, obtain fifth feature figure；
S231 exports object detection results and predeterminable area testing result, the preset areas according to the fifth feature figure
Domain is the region that robot enters that elevator needs to occupy；
S300 according to the object detection results and predeterminable area testing result, judge whether can enter the elevator, if
It is then to enter the elevator, if it is not, then discharging the elevator, and call elevator again.
Specifically, in the network architecture, usually as depth increase can reduce resolution ratio and improve port number.Such as Fig. 5 institute
Show, it is assumed that the resolution ratio of each characteristic pattern of the last layer extracted from mobilenetv3 is the 1/32 of original image, by last
Each characteristic pattern of layer is defined as fisrt feature figure；Then each characteristic pattern of 1/16 that layer that resolution ratio is original image is chosen again,
Each characteristic pattern of this layer is defined as second feature figure.
Then according to default step, fisrt feature figure is handled, makes one times of increase resolution of fisrt feature figure, obtains
To third feature figure, the resolution ratio of third feature figure is the 1/16 of original image.
Second feature figure is subjected to 1 × 1 convolution transform again, keeps second feature figure identical as the port number of third feature figure,
Then by after convolution transform second feature figure and third feature figure merge, obtain fourth feature figure.
2 Sub-reso promotions processing is carried out to fourth feature figure according still further to above-mentioned default step, obtains fifth feature figure,
The resolution ratio of the obtained fifth feature figure is the 1/4 of original image.Pass through Objects as Points after obtaining fifth feature figure
Then target in foundation structure detection image carries out territorial classification detection by the region decision branch of addition.
It is to each point minute in the fifth feature figure when carrying out territorial classification detection by fifth feature figure
Class, judges whether each point is occupied, and in fifth feature figure a point corresponds to one 4 × 4 region in original image, is equivalent to
Original image is divided into one by one 4 × 4 region, classifies to each point in fifth feature figure, is equivalent to in original image
Each 4 × 4 region carries out classification and Detection, and then show that the region in original image is occupied situation, finally from the detection knot of original image
The case where predeterminable area is occupied is obtained in fruit, and then obtains target area testing result.
Preferably, S221 is handled the fisrt feature figure according to default step, makes point of the fisrt feature figure
Resolution promotes one times, obtains third feature figure and specifically includes:
Deconvolution is carried out to the fisrt feature figure；
Channel separation is carried out to the fisrt feature figure after deconvolution；
Process of convolution is carried out to the fisrt feature figure after channel separation using two different convolution kernels；
Channel fusion is carried out to the characteristic pattern obtained after process of convolution, obtains third feature figure.
Specifically, default step includes deconvolution, channel separation, process of convolution and channel fusion.Its specific steps such as Fig. 5
It is shown.Deconvolution first is carried out to the 1/32 fisrt feature figure that resolution ratio is original image, then carries out channel separation, then carry out respectively
3 × 3 convolution sum, 5 × 5 convolution, finally carries out channel fusion again, obtains third feature figure, the resolution ratio of obtained third feature figure
One times is promoted, the resolution ratio of third feature figure is the 1/16 of original image.
It will be merged after second feature figure 1 × 1 process of convolution of progress with third feature figure, obtained fourth feature figure
Resolution ratio is also the 1/16 of original image.Then the resolution ratio carried out twice according still further to above-mentioned default step to fourth feature figure mentions
It rises, the resolution ratio of obtained fifth feature figure is the 1/4 of original image.By by the 1/ of the increase resolution of fifth feature figure to original image
4, keep the region segmentation carried out to original image more reasonable, avoid because region segmentation is excessive or it is too small due to it is influence area classification and Detection
Accuracy rate, while detection of the algorithm Objects as Points to target can also be taken into account.
It should be understood that in the above embodiments, the size of each step number is not meant that the order of the execution order, it is each to walk
Rapid execution sequence should determine that the implementation process without coping with the embodiment of the present invention constitutes any limit with function and internal logic
It is fixed.
The 5th embodiment provided according to the present invention, as shown in fig. 6, the system that elevator is taken by a kind of robot, comprising:
Image collection module 10, for obtaining the image of elevator interior when elevator door is opened；
Detection module 20 obtains target detection for inputting described image in trained neural network model in advance
As a result with predeterminable area testing result, the predeterminable area is the region that robot enters that elevator needs to occupy；
Processing module 30, for judging whether to enter according to the object detection results and predeterminable area testing result
The elevator, if so, if it is not, then discharging the elevator, and calling elevator again into the elevator.
Specifically, the present embodiment is the corresponding Installation practice of above method embodiment, specific effect is referring to above-mentioned implementation
Example, this is no longer going to repeat them.
The sixth embodiment provided according to the present invention, a kind of system that elevator is taken by robot, in above-mentioned 5th embodiment
On the basis of, processing module 30 includes:
Processing unit, for judging whether the predeterminable area is occupied according to the predeterminable area testing result, if
It is no, then determine that the elevator can be entered；
Processing unit, if being also used to judge that the predeterminable area is occupied, then root according to the predeterminable area testing result
According to the object detection results, judgement occupies whether object is object；
Processing unit, if being also used to the object that occupies is object, judgement can not enter the elevator；
Processing unit exports evacuation information if being also used to the object that occupies is not object, after waiting preset time, then
The secondary new images for obtaining the elevator interior, the new images are inputted in the neural network model, obtain target inspection again
Result and predeterminable area testing result are surveyed, and according to the object detection results and predeterminable area testing result obtained again, judgement
Whether can enter the elevator, when repeating to obtain the number of image of the elevator interior greater than preset threshold, then determine not
The elevator can be entered.
Specifically, the present embodiment is the corresponding Installation practice of above method embodiment, specific effect is referring to above-mentioned implementation
Example, this is no longer going to repeat them.
The 7th embodiment provided according to the present invention, a kind of system that elevator is taken by robot, in above-mentioned 5th embodiment
Or on the basis of sixth embodiment, detection module 20 includes:
Feature extraction unit extracts difference for inputting described image in trained neural network model in advance
The characteristic pattern of layer；
Increase resolution unit carries out Fusion Features and increase resolution for the characteristic pattern to different layers；
Task output unit, for exporting target detection according to the characteristic pattern obtained after Fusion Features and increase resolution
As a result with predeterminable area testing result.
Specifically, the present embodiment is the corresponding Installation practice of above method embodiment, specific effect is referring to above-mentioned implementation
Example, this is no longer going to repeat them.
The 8th embodiment provided according to the present invention, a kind of system that elevator is taken by robot, in above-mentioned 7th embodiment
On the basis of,
Feature extraction unit is also used to input described image in trained neural network model in advance, extract most
The fisrt feature figure of later layer, and extract the second feature figure of front layer, the resolution ratio of the second feature figure are described the
Twice of the resolution ratio of one characteristic pattern；
Increase resolution unit includes:
Increase resolution subelement, for handling the fisrt feature figure, making described first according to default step
One times of the increase resolution of characteristic pattern, obtains third feature figure；
Fusion Features subelement, for by the second feature figure carry out 1 × 1 process of convolution after with the third feature figure
It is merged, obtains fourth feature figure；
Increase resolution subelement will for repeatedly being handled the fourth feature figure according to the default step
More times of the increase resolution of the fourth feature figure, obtains fifth feature figure；
Task output unit is also used to export object detection results and predeterminable area detection according to the fifth feature figure
As a result.
Preferably, increase resolution subelement is also used to carry out deconvolution to the fisrt feature figure；After deconvolution
The fisrt feature figure carries out channel separation；Using two different convolution kernels to the fisrt feature figure after channel separation into
Row process of convolution；Channel fusion is carried out to the characteristic pattern obtained after process of convolution, obtains third feature figure.
Specifically, the present embodiment is the corresponding Installation practice of above method embodiment, specific effect is referring to above-mentioned implementation
Example, this is no longer going to repeat them.
It should be noted that above-described embodiment can be freely combined as needed.The above is only of the invention preferred
Embodiment, it is noted that for those skilled in the art, in the premise for not departing from the principle of the invention
Under, several improvements and modifications can also be made, these modifications and embellishments should also be considered as the scope of protection of the present invention.
Claims (10)
1. a kind of method that elevator is taken by robot characterized by comprising
When elevator door is opened, the image of elevator interior is obtained；
Described image is inputted in trained neural network model in advance, object detection results and predeterminable area detection knot are obtained
Fruit, the predeterminable area are the region that robot enters that elevator needs to occupy；
According to the object detection results and predeterminable area testing result, judge whether that the elevator can be entered, if so, into
The elevator if it is not, then discharging the elevator, and calls elevator again.
2. the method that elevator is taken by a kind of robot according to claim 1, which is characterized in that described according to the target
Testing result and predeterminable area testing result, the elevator can be entered by, which judging whether, specifically includes:
According to the predeterminable area testing result, judge whether the predeterminable area is occupied, if it is not, then determining to enter described
Elevator；
If judging that the predeterminable area is occupied according to the predeterminable area testing result, then according to the object detection results,
Judgement occupies whether object is object；
If the object that occupies is object, judgement can not enter the elevator；
If the object that occupies is not object, evacuation information is exported, after waiting preset time, obtains the elevator interior again
New images input the new images in the neural network model, obtain object detection results and predeterminable area detection again
As a result, simultaneously according to the object detection results and predeterminable area testing result obtained again judge whether that the elevator can be entered, when
When the number that repetition obtains the image of the elevator interior is greater than preset threshold, then determine that the elevator can not be entered.
3. the method that elevator is taken by a kind of robot according to claim 1, which is characterized in that described that described image is defeated
Enter in preparatory trained neural network model, obtain object detection results and predeterminable area testing result specifically includes:
Described image is inputted in trained neural network model in advance, the characteristic pattern of different layers is extracted；
Fusion Features and increase resolution are carried out to the characteristic pattern of different layers；
According to the characteristic pattern obtained after Fusion Features and increase resolution, object detection results and predeterminable area detection knot are exported
Fruit.
4. the method that elevator is taken by a kind of robot according to claim 3, which is characterized in that described that described image is defeated
Enter in preparatory trained neural network model, the characteristic pattern for extracting different layers specifically includes:
Described image is inputted in trained neural network model in advance, extracts the fisrt feature figure of the last layer, and mention
The second feature figure of front layer is taken out, the resolution ratio of the second feature figure is twice of the resolution ratio of the fisrt feature figure；
The characteristic pattern to different layers carries out Fusion Features and increase resolution specifically includes:
According to default step, the fisrt feature figure is handled, makes one times of increase resolution of the fisrt feature figure, obtains
To third feature figure；
It will be merged after the second feature figure 1 × 1 process of convolution of progress with the third feature figure, obtain fourth feature
Figure；
According to the default step, the fourth feature figure is repeatedly handled, the resolution ratio of the fourth feature figure is mentioned
More times are risen, fifth feature figure is obtained；
It is described to be detected according to the characteristic pattern obtained after Fusion Features and increase resolution, output object detection results and predeterminable area
As a result it specifically includes:
According to the fifth feature figure, object detection results and predeterminable area testing result are exported.
5. the method that elevator is taken by a kind of robot according to claim 4, which is characterized in that described according to default step
Suddenly, the fisrt feature figure is handled, makes one times of increase resolution of the fisrt feature figure, obtain third feature figure tool
Body includes:
Deconvolution is carried out to the fisrt feature figure；
Channel separation is carried out to the fisrt feature figure after deconvolution；
Process of convolution is carried out to the fisrt feature figure after channel separation using two different convolution kernels；
Channel fusion is carried out to the characteristic pattern obtained after process of convolution, obtains third feature figure.
6. the system that elevator is taken by a kind of robot characterized by comprising
Image collection module, for obtaining the image of elevator interior when elevator door is opened；
Detection module, for described image to be inputted in advance in trained neural network model, obtain object detection results and
Predeterminable area testing result, the predeterminable area are the region that robot enters that elevator needs to occupy；
Processing module, for judging whether that the electricity can be entered according to the object detection results and predeterminable area testing result
Ladder, if so, if it is not, then discharging the elevator, and calling elevator again into the elevator.
7. the system that elevator is taken by a kind of robot according to claim 6, which is characterized in that the processing module packet
It includes:
Processing unit, for judging whether the predeterminable area is occupied, if it is not, then according to the predeterminable area testing result
Judgement can enter the elevator；
The processing unit, if being also used to judge that the predeterminable area is occupied, then root according to the predeterminable area testing result
According to the object detection results, judgement occupies whether object is object；
The processing unit, if being also used to the object that occupies is object, judgement can not enter the elevator；
The processing unit exports evacuation information if being also used to the object that occupies is not object, after waiting preset time, then
The secondary new images for obtaining the elevator interior, the new images are inputted in the neural network model, obtain target inspection again
Result and predeterminable area testing result are surveyed, and according to the object detection results and predeterminable area testing result obtained again, judgement
Whether can enter the elevator, when repeating to obtain the number of image of the elevator interior greater than preset threshold, then determine not
The elevator can be entered.
8. the system that elevator is taken by a kind of robot according to claim 6, which is characterized in that the detection module packet
It includes:
Feature extraction unit extracts different layers for inputting described image in trained neural network model in advance
Characteristic pattern；
Increase resolution unit carries out Fusion Features and increase resolution for the characteristic pattern to different layers；
Task output unit, for exporting object detection results according to the characteristic pattern obtained after Fusion Features and increase resolution
With predeterminable area testing result.
9. the system that elevator is taken by a kind of robot according to claim 8, which is characterized in that
The feature extraction unit is also used to input described image in trained neural network model in advance, extract most
The fisrt feature figure of later layer, and extract the second feature figure of front layer, the resolution ratio of the second feature figure are described the
Twice of the resolution ratio of one characteristic pattern；
The increase resolution unit includes:
Increase resolution subelement, for handling the fisrt feature figure, making the fisrt feature according to default step
One times of the increase resolution of figure, obtains third feature figure；
Fusion Features subelement, for will be carried out after the second feature figure 1 × 1 process of convolution of progress with the third feature figure
Fusion, obtains fourth feature figure；
The increase resolution subelement will for repeatedly being handled the fourth feature figure according to the default step
More times of the increase resolution of the fourth feature figure, obtains fifth feature figure；
The task output unit is also used to export object detection results and predeterminable area detection according to the fifth feature figure
As a result.
10. the system that elevator is taken by a kind of robot according to claim 9, which is characterized in that
The increase resolution subelement is also used to carry out deconvolution to the fisrt feature figure；To described after deconvolution
One characteristic pattern carries out channel separation；Convolution is carried out to the fisrt feature figure after channel separation using two different convolution kernels
Processing；Channel fusion is carried out to the characteristic pattern obtained after process of convolution, obtains third feature figure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910551001.8A CN110276302B (en) | 2019-06-24 | 2019-06-24 | Method and system for taking elevator by robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910551001.8A CN110276302B (en) | 2019-06-24 | 2019-06-24 | Method and system for taking elevator by robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110276302A true CN110276302A (en) | 2019-09-24 |
| CN110276302B CN110276302B (en) | 2022-11-25 |
Family
ID=67961774
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910551001.8A Active CN110276302B (en) | 2019-06-24 | 2019-06-24 | Method and system for taking elevator by robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110276302B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111161740A (en) * | 2019-12-31 | 2020-05-15 | 中国建设银行股份有限公司 | Intention recognition model training method, intention recognition method and related device |
| CN111153300A (en) * | 2019-12-31 | 2020-05-15 | 深圳优地科技有限公司 | Ladder taking method and system for robot, robot and storage medium |
| CN111847152A (en) * | 2020-06-30 | 2020-10-30 | 深圳优地科技有限公司 | Robot elevator taking determination method and device, electronic equipment and medium |
| CN111874764A (en) * | 2020-09-28 | 2020-11-03 | 上海木承智能医疗科技有限公司 | Robot scheduling method, server and storage medium |
| CN112529953A (en) * | 2020-12-17 | 2021-03-19 | 深圳市普渡科技有限公司 | Elevator space state judgment method and device and storage medium |
| CN112537705A (en) * | 2020-03-31 | 2021-03-23 | 深圳优地科技有限公司 | Robot elevator taking scheduling method and device, terminal equipment and storage medium |
| CN113911864A (en) * | 2021-10-13 | 2022-01-11 | 北京云迹科技有限公司 | Control method for robot to board elevator and related equipment |
| US11513522B2 (en) * | 2019-11-22 | 2022-11-29 | Lg Electronics Inc. | Robot using an elevator and method for controlling the same |
| CN115893138A (en) * | 2022-10-27 | 2023-04-04 | 合肥瓦力觉启机器人科技有限公司 | Wireless interaction system in AGV dolly elevator car |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005018382A (en) * | 2003-06-25 | 2005-01-20 | Matsushita Electric Works Ltd | Autonomous mobile robot for getting on and off elevator |
| JP2011088721A (en) * | 2009-10-23 | 2011-05-06 | Fuji Heavy Ind Ltd | Autonomous travel robot and control system of autonomous travel robot |
| US20120041593A1 (en) * | 2010-07-08 | 2012-02-16 | Ryoko Ichinose | Elevator system that autonomous mobile robot takes together with person |
| CN102821272A (en) * | 2012-08-16 | 2012-12-12 | 安徽中科智能高技术有限责任公司 | Video monitoring system with elevator invalid request signal removing function |
| CN106965198A (en) * | 2017-03-30 | 2017-07-21 | 上海木爷机器人技术有限公司 | Robot control method and device |
| CN108439099A (en) * | 2018-04-26 | 2018-08-24 | 珠海亿智电子科技有限公司 | A kind of elevator intelligent control method based on space and load detecting |
| CN109059882A (en) * | 2018-08-07 | 2018-12-21 | 北京云迹科技有限公司 | Interior space detection method and system |
| CN109095299A (en) * | 2018-08-23 | 2018-12-28 | 北京云迹科技有限公司 | Robot boarding method and device based on Internet of Things |
| CN109867180A (en) * | 2017-12-01 | 2019-06-11 | 株式会社日立大厦*** | Elevator device |
-
2019
- 2019-06-24 CN CN201910551001.8A patent/CN110276302B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005018382A (en) * | 2003-06-25 | 2005-01-20 | Matsushita Electric Works Ltd | Autonomous mobile robot for getting on and off elevator |
| JP2011088721A (en) * | 2009-10-23 | 2011-05-06 | Fuji Heavy Ind Ltd | Autonomous travel robot and control system of autonomous travel robot |
| US20120041593A1 (en) * | 2010-07-08 | 2012-02-16 | Ryoko Ichinose | Elevator system that autonomous mobile robot takes together with person |
| CN102821272A (en) * | 2012-08-16 | 2012-12-12 | 安徽中科智能高技术有限责任公司 | Video monitoring system with elevator invalid request signal removing function |
| CN106965198A (en) * | 2017-03-30 | 2017-07-21 | 上海木爷机器人技术有限公司 | Robot control method and device |
| CN109867180A (en) * | 2017-12-01 | 2019-06-11 | 株式会社日立大厦*** | Elevator device |
| CN108439099A (en) * | 2018-04-26 | 2018-08-24 | 珠海亿智电子科技有限公司 | A kind of elevator intelligent control method based on space and load detecting |
| CN109059882A (en) * | 2018-08-07 | 2018-12-21 | 北京云迹科技有限公司 | Interior space detection method and system |
| CN109095299A (en) * | 2018-08-23 | 2018-12-28 | 北京云迹科技有限公司 | Robot boarding method and device based on Internet of Things |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11513522B2 (en) * | 2019-11-22 | 2022-11-29 | Lg Electronics Inc. | Robot using an elevator and method for controlling the same |
| CN111153300A (en) * | 2019-12-31 | 2020-05-15 | 深圳优地科技有限公司 | Ladder taking method and system for robot, robot and storage medium |
| CN111161740A (en) * | 2019-12-31 | 2020-05-15 | 中国建设银行股份有限公司 | Intention recognition model training method, intention recognition method and related device |
| CN111153300B (en) * | 2019-12-31 | 2022-01-07 | 深圳优地科技有限公司 | Ladder taking method and system for robot, robot and storage medium |
| CN112537705B (en) * | 2020-03-31 | 2023-04-11 | 深圳优地科技有限公司 | Robot elevator taking scheduling method and device, terminal equipment and storage medium |
| CN112537705A (en) * | 2020-03-31 | 2021-03-23 | 深圳优地科技有限公司 | Robot elevator taking scheduling method and device, terminal equipment and storage medium |
| CN111847152A (en) * | 2020-06-30 | 2020-10-30 | 深圳优地科技有限公司 | Robot elevator taking determination method and device, electronic equipment and medium |
| CN111874764A (en) * | 2020-09-28 | 2020-11-03 | 上海木承智能医疗科技有限公司 | Robot scheduling method, server and storage medium |
| CN112529953B (en) * | 2020-12-17 | 2022-05-03 | 深圳市普渡科技有限公司 | Elevator space state judgment method and device and storage medium |
| WO2022127451A1 (en) * | 2020-12-17 | 2022-06-23 | 深圳市普渡科技有限公司 | Method and apparatus for determining spatial state of elevator, and storage medium |
| CN112529953A (en) * | 2020-12-17 | 2021-03-19 | 深圳市普渡科技有限公司 | Elevator space state judgment method and device and storage medium |
| CN113911864A (en) * | 2021-10-13 | 2022-01-11 | 北京云迹科技有限公司 | Control method for robot to board elevator and related equipment |
| CN115893138A (en) * | 2022-10-27 | 2023-04-04 | 合肥瓦力觉启机器人科技有限公司 | Wireless interaction system in AGV dolly elevator car |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110276302B (en) | 2022-11-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110276302A (en) | The method and system of elevator are taken by a kind of robot | |
| CN110348376B (en) | Pedestrian real-time detection method based on neural network | |
| CN104809443B (en) | Detection method of license plate and system based on convolutional neural networks | |
| CN111553387B (en) | Personnel target detection method based on Yolov3 | |
| CN110751099B (en) | Unmanned aerial vehicle aerial video track high-precision extraction method based on deep learning | |
| CN108549841A (en) | A kind of recognition methods of the Falls Among Old People behavior based on deep learning | |
| CN107229929A (en) | A kind of license plate locating method based on R CNN | |
| CN110147763A (en) | Video semanteme dividing method based on convolutional neural networks | |
| CN109559302A (en) | Pipe video defect inspection method based on convolutional neural networks | |
| CN108776819A (en) | A kind of target identification method, mobile terminal and computer readable storage medium | |
| CN105574550A (en) | Vehicle identification method and device | |
| CN112417990B (en) | Examination student illegal behavior identification method and system | |
| CN109636846B (en) | Target positioning method based on cyclic attention convolution neural network | |
| CN113537099A (en) | Dynamic detection method for fire smoke in highway tunnel | |
| CN108039044A (en) | The system and method that Vehicular intelligent based on multiple dimensioned convolutional neural networks is lined up | |
| CN107273852A (en) | Escalator floor plates object and passenger behavior detection algorithm based on machine vision | |
| CN101364263A (en) | Method and system for detecting skin texture to image | |
| CN112733815B (en) | Traffic light identification method based on RGB outdoor road scene image | |
| CN109472226A (en) | A kind of sleep behavioral value method based on deep learning | |
| CN112818871B (en) | Target detection method of full fusion neural network based on half-packet convolution | |
| CN111223129A (en) | Detection method, detection device, monitoring equipment and computer readable storage medium | |
| CN109697449A (en) | A kind of object detection method, device and electronic equipment | |
| CN111401188A (en) | Traffic police gesture recognition method based on human body key point features | |
| CN108460336A (en) | A kind of pedestrian detection method based on deep learning | |
| EP0435660B1 (en) | Method of evaluating objects based upon image processing, and inspection apparatus using said method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 200335 402 rooms, No. 33, No. 33, Guang Shun Road, Shanghai Patentee after: Shanghai Zhihui Medical Technology Co.,Ltd. Address before: 200335 402 rooms, No. 33, No. 33, Guang Shun Road, Shanghai Patentee before: SHANGHAI MROBOT TECHNOLOGY Co.,Ltd. Address after: 200335 402 rooms, No. 33, No. 33, Guang Shun Road, Shanghai Patentee after: Shanghai zhihuilin Medical Technology Co.,Ltd. Address before: 200335 402 rooms, No. 33, No. 33, Guang Shun Road, Shanghai Patentee before: Shanghai Zhihui Medical Technology Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| CP03 | Change of name, title or address |
Address after: 202150 room 205, zone W, second floor, building 3, No. 8, Xiushan Road, Chengqiao Town, Chongming District, Shanghai (Shanghai Chongming Industrial Park) Patentee after: Shanghai Noah Wood Robot Technology Co.,Ltd. Address before: 200335 402 rooms, No. 33, No. 33, Guang Shun Road, Shanghai Patentee before: Shanghai zhihuilin Medical Technology Co.,Ltd. |
|
| CP03 | Change of name, title or address | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230804 Address after: 610, Floor 6, Block A, No. 2, Lize Middle Second Road, Chaoyang District, Beijing 100102 Patentee after: Zhongguancun Technology Leasing Co.,Ltd. Address before: 202150 room 205, zone W, second floor, building 3, No. 8, Xiushan Road, Chengqiao Town, Chongming District, Shanghai (Shanghai Chongming Industrial Park) Patentee before: Shanghai Noah Wood Robot Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |