CN110378246A - Ground detection method, apparatus, computer readable storage medium and electronic equipment - Google Patents
Ground detection method, apparatus, computer readable storage medium and electronic equipment Download PDFInfo
- Publication number
- CN110378246A CN110378246A CN201910563488.1A CN201910563488A CN110378246A CN 110378246 A CN110378246 A CN 110378246A CN 201910563488 A CN201910563488 A CN 201910563488A CN 110378246 A CN110378246 A CN 110378246A
- Authority
- CN
- China
- Prior art keywords
- ground
- ordered
- point cloud
- dimensional point
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
This disclosure relates to a kind of ground detection method, apparatus, computer readable storage medium and electronic equipment.This method comprises: the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image of acquisition target floor；Using ordered three-dimensional point cloud as the input of ground detection model trained in advance, obtain the corresponding ground detection of target floor as a result, ground detection result includes floor type, the floor type be used to characterize target floor whether be slope slope ground.In this way, by the way that the ordered three-dimensional point cloud is input to ground detection model, the corresponding ground detection of the target floor can be obtained as a result, without using artificial detection algorithm, logic complicated, robustness bad problem high so as to avoid artificial algorithm design difficulty.Also, due to being detected according to ordered three-dimensional point cloud to ground, so, can to avoid disparity map in the related technology and depth map in overexposure, the drawbacks of will appear more empty, biggish error under excessively dark, weak texture region.
Description
Technical field
This disclosure relates to technical field of computer vision, and in particular, to a kind of ground detection method, apparatus, computer
Readable storage medium storing program for executing and electronic equipment.
Background technique
In fields such as guide, robot, automatic Pilots, slope detection is an extremely important Technology Difficulties.People visually impaired
Scholar, robot, automatic driving vehicle need to predict the floor type on front ground in advance, for example, the floor type on front ground
No is upward slope type, descending type or horizontal type, in this way, Different Ground type could be directed to by adjusting traveling appearance in advance
State and mode are advanced safely to ensure.
Level ground, slope ground cannot be distinguished in the ground detection method based on RGB two dimensional image at all in the related technology,
Therefore differentiation level ground is involved the need for, the ground detection of slope ground is normally based on disparity map, depth map to carry out.And
The defect of disparity map and depth map due to obtaining principle usually will appear more sky in overexposure, under excessively dark, weak texture region
Hole, biggish error, therefore the artificial algorithm design difficulty based on such data is high, logic is complicated, robustness is bad, is difficult
Obtain accurate floor type, so cannot accurately for front ground floor type by adjustment traveling posture in advance and
Mode is advanced safely to ensure.
Summary of the invention
Purpose of this disclosure is to provide a kind of ground detection method, apparatus, computer readable storage medium and electronic equipment,
Ground is accurately detected with realizing.
To achieve the goals above, the disclosure provides a kind of ground detection method, comprising:
Obtain the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image of target floor；
Using the ordered three-dimensional point cloud as the input of ground detection model trained in advance, the target floor pair is obtained
The ground detection answered is as a result, the ground detection result includes floor type, and the floor type is for characterizing the target
Whether face is the slope ground comprising slope.
Optionally, the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image for obtaining target floor, packet
It includes:
Obtain the posture under the corresponding depth map of the ground image and the corresponding world coordinate system of the ground image
Angle, the attitude angle include the angle for obtaining the image acquiring device of the ground image under world coordinate system；
According to the depth map and the attitude angle, orderly three under the corresponding world coordinate system of the ground image are determined
Dimension point cloud.
Optionally, described according to the depth map and the attitude angle, determine the corresponding world coordinates of the ground image
Ordered three-dimensional point cloud under system, comprising:
According to the depth map and preset Normalized Scale, normalization factor is determined；
Depth map according to the depth map and the normalization factor, after determining normalization；
According to after the normalization depth map and the attitude angle, determine the corresponding world coordinate system of the ground image
Under ordered three-dimensional point cloud.
Optionally, if it is the slope ground, the ground detection result that the floor type, which characterizes the target floor,
Further include: the location information of inclination angle and the slope in the ground image of the slope with respect to the horizontal plane.
Optionally, training obtains the ground detection model in the following manner:
Obtain the sample ground image on sample ground；
The sample ordered three-dimensional point cloud under the corresponding world coordinate system of the sample ground image is obtained, the sample is orderly
Three-dimensional point cloud be labeled with the sample floor type on the sample ground, the sample inclination angle of the sample ground with respect to the horizontal plane with
And sample position of the sample ground in the sample ground image；
It is trained using the sample ordered three-dimensional point cloud as model training sample, obtains the ground detection model.
Optionally, the method also includes:
According to ordered three-dimensional point of the slope under the location information and the world coordinate system in the ground image
Cloud determines location information of the slope under the world coordinate system.
Optionally, the method also includes:
If it is the slope ground that the floor type, which characterizes the target floor, prompt information is exported.
The disclosure also provides a kind of ground detection, comprising:
First obtains module, the ordered three-dimensional point under the corresponding world coordinate system of ground image for obtaining target floor
Cloud；
Input module, for obtaining using the ordered three-dimensional point cloud as the input of ground detection model trained in advance
As a result, the ground detection result includes floor type, the floor type is used for the corresponding ground detection of the target floor
Characterize whether the target floor is the slope ground comprising slope.
Optionally, the first acquisition module includes:
Acquisition submodule, for obtaining the corresponding depth map of the ground image and the corresponding generation of the ground image
Attitude angle under boundary's coordinate system, the attitude angle include obtaining the image acquiring device of the ground image under world coordinate system
Angle；
Submodule is determined, for determining the corresponding world of the ground image according to the depth map and the attitude angle
Ordered three-dimensional point cloud under coordinate system.
Optionally, the determining submodule includes:
Normalization factor determines submodule, for determining normalization according to the depth map and preset Normalized Scale
The factor；
Depth map determines submodule, for the depth according to the depth map and the normalization factor, after determining normalization
Degree figure；
Point cloud determine submodule, for according to after the normalization depth map and the attitude angle, determine the ground
Ordered three-dimensional point cloud under the corresponding world coordinate system of image.
Optionally, if it is the slope ground, the ground detection result that the floor type, which characterizes the target floor,
Further include: the location information of inclination angle and the slope in the ground image of the slope with respect to the horizontal plane.
Optionally, described device further include:
Second obtains module, for obtaining the sample ground image on sample ground；
Third obtains module, for obtaining the sample ordered three-dimensional under the corresponding world coordinate system of the sample ground image
Point cloud, the sample ordered three-dimensional point cloud be labeled with the sample floor type on the sample ground, the sample ground relative to
The sample position of the sample inclination angle of horizontal plane and the sample ground in the sample ground image；
Training module obtains described for being trained using the sample ordered three-dimensional point cloud as model training sample
Ground detection model.
Optionally, described device further include:
Determining module, for according to the slope under the location information and the world coordinate system in the ground image
Ordered three-dimensional point cloud, determine location information of the slope under the world coordinate system.
Optionally, described device further include:
Output module exports prompt if characterizing the target floor for the floor type is the slope ground
Information.
The disclosure also provides a kind of computer readable storage medium, is stored thereon with computer program, and the program is processed
The step of ground detection method provided by the disclosure is realized when device executes.
The disclosure also provides a kind of electronic equipment, comprising:
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize ground provided by the disclosure
The step of detection method.
Through the above technical solutions, the ordered three-dimensional point under the corresponding world coordinate system of ground image of acquisition target floor
Cloud；Using ordered three-dimensional point cloud as the input of ground detection model trained in advance, the corresponding ground detection of target floor is obtained
As a result, ground detection result includes floor type, the floor type be used to characterize target floor whether be slope slope ground.
In this way, can be obtained by the way that the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image is input to ground detection model
To the corresponding ground detection of the target floor as a result, without using artificial detection algorithm, it is difficult to design so as to avoid artificial algorithm
Spend the problem high, logic is complicated, robustness is bad.Also, due to according to having under the corresponding world coordinate system of ground image
Sequence three-dimensional point cloud detects target floor, so, it can be to avoid disparity map in the related technology and depth map in overexposure, mistake
Secretly, the drawbacks of will appear more empty, biggish error under weak texture region.In addition, the ground detection model is based on existing
What the neural network framework training of some RGB two dimensional images obtained, in this way, without constructing new neural network framework, using now
The detection to ground can be realized in some neural network frameworks.
Other feature and advantage of the disclosure will the following detailed description will be given in the detailed implementation section.
Detailed description of the invention
Attached drawing is and to constitute part of specification for providing further understanding of the disclosure, with following tool
Body embodiment is used to explain the disclosure together, but does not constitute the limitation to the disclosure.In the accompanying drawings:
Fig. 1 is a kind of flow chart of ground detection method shown according to an exemplary embodiment.
Fig. 2 is a kind of process for obtaining the ordered three-dimensional point cloud under world coordinate system shown according to an exemplary embodiment
Figure.
Fig. 3 is a kind of block diagram of ground detection shown according to an exemplary embodiment.
Fig. 4 is the block diagram of a kind of electronic equipment shown according to an exemplary embodiment.
Fig. 5 is the block diagram of a kind of electronic equipment shown according to another exemplary embodiment.
Specific embodiment
It is described in detail below in conjunction with specific embodiment of the attached drawing to the disclosure.It should be understood that this place is retouched
The specific embodiment stated is only used for describing and explaining the disclosure, is not limited to the disclosure.
The neural network framework of the deep learning in computer vision field is divided into two classes: one kind is based on RGB two dimension
The neural network framework of image, another kind of is the neural network framework (for example, PointNet) based on unordered three-dimensional point cloud.Due to
Unordered three-dimensional point cloud is to remove Null Spot cloud only to retain the point cloud of available point cloud, and the position of unordered three-dimensional point cloud midpoint cloud can be with
Meaning change, the image alignment that can not be obtained with visual sensor, so, technical staff can not be based on institute's acquired image at this
It is labeled in unordered three-dimensional point cloud.The input of neural network framework based on RGB two dimensional image is that triple channel specifically can
With by the image real time transfer of RGB two dimensional image be the corresponding image data of R R matrix, the G matrix of the corresponding image data of G,
Then R matrix, G matrix, B matrix are separately input into triple channel by the B matrix of the corresponding image data of B.
The image alignment that inventor can obtain in view of ordered three-dimensional point cloud with visual sensor, and then can be according to figure
As being labeled in ordered three-dimensional point cloud, and, ordered three-dimensional point cloud can also be indicated with three-dimensional matrice, e.g., X matrix, Y
Therefore matrix and Z matrix both may be implemented to mark using ordered three-dimensional point cloud, can also be compatible with existing based on RGB X-Y scheme
The neural network framework (neural network framework that input is triple channel) of picture, no longer needs to that other neural network frames are additionally arranged
Frame.It is therefore proposed that a kind of ground detection method, apparatus, computer readable storage medium and mobile device.
Fig. 1 is a kind of flow chart of ground detection method shown according to an exemplary embodiment.As shown in Figure 1, the party
Method the following steps are included:
In a step 11, the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image of target floor is obtained.
Ordered three-dimensional point cloud under world coordinate system can be the laser point cloud obtained based on laser radar, can also be based on
The point cloud that ground image obtains.Since the point cloud obtained based on ground image is aligned with ground image, there is density, after
The accuracy that ground detection can be improved when the continuous progress ground detection based on the cloud avoids missing inspection, and is obtained based on ground image
The point cloud obtained can be indicated with three-dimensional matrice, can be used as the input of the existing neural network framework based on RGB two dimensional image,
Without additionally constructing other neural network frameworks.Therefore, in the disclosure, by taking the point cloud obtained based on ground image as an example,
It is illustrated.
Specifically, the ground image of the target floor can be obtained by image acquiring device.Image acquiring device can be with
For visual sensors such as monocular camera, binocular camera, depth cameras.Target floor can be image acquiring device and can get
Maximum magnitude in ground, be also possible to the ground in the target zone of user setting.In one embodiment, this method can
To be applied to robot device, the mobile mobile devices such as the helmet and automatic driving vehicle, which can be installed
On the mobile device, when image acquiring device takes the ground image of target floor, which is sent to shifting
Dynamic equipment is handled, to obtain the ordered three-dimensional point cloud under the corresponding world coordinate system of the ground image.In another embodiment
In, this method can also be applied to server, when image acquiring device gets the ground image of target floor, by the ground
Image is sent to server and is handled, to obtain the ordered three-dimensional point cloud under the corresponding world coordinate system of the ground image.
In step 12, using ordered three-dimensional point cloud as the input of ground detection model trained in advance, with obtaining target
The corresponding ground detection result in face.
After obtaining the ordered three-dimensional point cloud under world coordinate system, as ground detection model trained in advance
Input, can be obtained the corresponding ground detection result of target floor.Wherein, ground detection result includes floor type, the ground
Type is for characterizing whether target floor is the slope ground comprising slope.
In the moving process such as guide, robot movement, automatic Pilot, need whether ground in front of real-time detection has tiltedly
Slope, so that visually impaired people, robot, automatic driving vehicle can predict the floor type on front ground in advance.Particularly important
It is that need to predict whether front ground has a slope in advance, it, can be by advance and when ground includes slope in front of precognition in advance
Traveling posture and mode are adjusted to ensure mobile security.Therefore, in the disclosure, which includes at least ground noodles
Type, the floor type is for characterizing whether target floor is the slope ground comprising slope.
In addition, as described above, the ground detection model is the neural network framework training based on existing RGB two dimensional image
It obtains, it can be such as are as follows: the neural network models such as RCNN, Fast-RCNN, Faster-RCNN, Yolo, SSD.In this way, nothing
New neural network framework need to be constructed, is trained using existing neural network framework, the detection to ground can be realized.
Through the above technical solutions, the ordered three-dimensional point under the corresponding world coordinate system of ground image of acquisition target floor
Cloud；Using ordered three-dimensional point cloud as the input of ground detection model trained in advance, the corresponding ground detection of target floor is obtained
As a result, ground detection result includes floor type, the floor type be used to characterize target floor whether be slope slope ground.
In this way, can be obtained by the way that the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image is input to ground detection model
To the corresponding ground detection of the target floor as a result, without using artificial detection algorithm, it is difficult to design so as to avoid artificial algorithm
Spend the problem high, logic is complicated, robustness is bad.Also, due to according to having under the corresponding world coordinate system of ground image
Sequence three-dimensional point cloud detects target floor, so, it can be to avoid disparity map in the related technology and depth map in overexposure, mistake
Secretly, the drawbacks of will appear more empty, biggish error under weak texture region.In addition, the ground detection model is based on existing
What the neural network framework training of some RGB two dimensional images obtained, in this way, without constructing new neural network framework, using now
The detection to ground can be realized in some neural network frameworks.
The ground detection method of disclosure offer is be provided for the ease of those skilled in the art, it is complete with one below
The ground detection method is described in whole embodiment.
As shown in Fig. 2, the step 11 in Fig. 1 may include step 111 and step 112.
In step 111, it obtains under the corresponding depth map of ground image and the corresponding world coordinate system of ground image
Attitude angle.Wherein, attitude angle includes angle of the image acquiring device of acquisition ground image under world coordinate system.
Specifically, depth map, which can be, directly passes through what the visual sensor, RGB-D camera etc. directly acquired,
It can be what the ground image post-processing got according to image acquiring device obtained, the depth map come in the disclosure
Source with no restrictions, as long as the depth map is corresponding with the ground image of target floor.Attitude angle is obtained by attitude transducer
It obtains, which includes angle of the image acquiring device of acquisition ground image under world coordinate system.
In step 112, according to depth map and attitude angle, orderly three under the corresponding world coordinate system of ground image are determined
Dimension point cloud.
In one embodiment, in order to improve computational efficiency, ground detection is quickly obtained as a result, getting depth map
After attitude angle, depth map is normalized, to obtain the depth map after normalized.
Specifically, specific embodiment depth map being normalized are as follows: firstly, according to depth map and preset
Normalized Scale determines normalization factor, illustratively, determines normalization factor according to formula (1):
Wherein, S characterizes normalization factor, and Norm characterizes preset Normalized Scale, keeps for each depth map value
Constant, W characterizes the width of depth map, and H characterizes the height of depth map.
Then, the depth map according to depth map and normalization factor, after determining normalization.It illustratively, can be according to formula
(2), the depth map after normalization is determined:
Wherein, WSThe width of depth map after characterization normalization, HSThe height of depth map after characterization normalization.Pass through WSAnd HS
Depth map after being assured that out normalization.
Then, based on the depth map and attitude angle after normalization, having under the corresponding world coordinate system of ground image is determined
Sequence three-dimensional point cloud.
Illustratively, by taking image acquiring device is camera as an example, according to the depth map after formula (3) and normalization, phase is constructed
Three-dimensional point cloud under machine coordinate system:
Wherein, u and v is the position coordinates of pixel in the depth map after normalization, M3×4It is the internal reference matrix of camera, Xc、
YcAnd ZcIt is coordinate value of the three-dimensional point cloud under camera coordinates system, ZcIt is the depth value of pixel in depth map after normalizing, is
Known quantity.
For each pixel in depth map, the seat under camera coordinates system is calculated using above-mentioned formula (3)
Scale value, according to the one-to-one mode of pixel in depth map or ground image, being from left to right arranged in from top to bottom has
Identical wide and high triple channel floating-point matrix, X is stored in three channels respectivelyc、YcAnd ZcValue, Null Spot with (0,0,0) replace,
Ordered three-dimensional point cloud can be built into.
Using camera photocentre as world coordinate system origin, choose is X horizontally to the rightWAxis positive direction is vertically downward YWAxis is square
To vertical XW、YWPlane is simultaneously directed straight ahead for ZWAxis positive direction establishes the world coordinate system of standard level posture.According to camera
Relational expression (4) between coordinate system and the world coordinate system of standard level posture, can be by the ordered three-dimensional under camera coordinates system
Point cloud P (Xc, Yc, Zc) be converted to ordered three-dimensional point cloud P (X under world coordinate systemW, YW, ZW):
Wherein, XW, YW, ZWThe coordinate value of the ordered three-dimensional point cloud under world coordinate system is characterized, α, β, γ characterize posture respectively
The angle of camera coordinates system X and horizontal plane, the projection of camera coordinates system Y-axis in the horizontal plane and world coordinate system Y-axis in angle
The angle of angle, camera coordinates system Z axis and the vertical guide including camera coordinates system X.
So far, the ordered three-dimensional point cloud under the corresponding world coordinate system of the ground image of available target floor.?
To after ordered three-dimensional point cloud, which is input in ground detection model trained in advance, to obtain target
The corresponding ground detection result in opposite.
In practical applications, there are when slope in detecting target floor, it is also necessary to further output Slope Facies for
The inclination angle and slope of horizontal plane location information in ground image.In this way, the processor or server of equipment easy to remove can
With the location information of inclination angle and slope in ground image further according to the slope with respect to the horizontal plane, adjustment movement is set
Standby traveling posture and mode is to guarantee safety of advancing.
It is obtained it should be noted that the ground detection model can train in the following manner:
Firstly, obtaining the sample ground image on sample ground.Wherein, which is including at least floor type
The image on the sample ground of Slop type, and the sample ground image is multiple.
Then, the ordered three-dimensional point cloud under the corresponding world coordinate system of sample ground image is obtained.As noted previously, as
The ground detection model is that the neural network framework based on existing RGB two dimensional image obtains, and the neural network framework has
There is triple channel input, and, technical staff is based on the position of slope, slope in sample ground image in sample ground image
It sets and slope inclination angle with respect to the horizontal plane is labeled sample point cloud, so, in the disclosure, need this
Sample ordered three-dimensional point cloud under the corresponding world coordinate system of sample ground image.
Wherein, which be aligned with sample ground image, so that technical staff be facilitated to be based on sample
Position and slope inclination angle with respect to the horizontal plane of the slope, slope that this ground image includes in sample ground image exists
It is labeled in the sample ordered three-dimensional point cloud, is labeled with sample floor type, sample ground with respect to the horizontal plane to obtain
The sample ordered three-dimensional point cloud of the sample position of sample inclination angle, sample ground in sample ground image.
Finally, being trained using the sample ordered three-dimensional point cloud as model training sample, to obtain ground detection model.
Illustratively, deep neural network model can be trained by the sample ordered three-dimensional point cloud, obtains ground
Detection model.In this way, the ordered three-dimensional point cloud under the corresponding world coordinate system of the ground image of target floor is input to this
When the ground detection model that training obtains in advance, the available corresponding ground detection of target floor is as a result, the testing result packet
Floor type is included, and, when floor type is slope ground, which can also include: the Slope Facies for level
The inclination angle in face and location information of the slope in ground image.
It should be noted that mobile device needs to adjust different traveling appearances due in upward slope ground and descending ground
State and mode therefore, need to also be into one if the floor type of target floor is Slop type in a kind of possible embodiment
It is upward slope type or descending type that step, which distinguishes the Slop type, that is, is removed in the ground detection result of ground detection model output
Be including characterization target floor slope ground floor type except, also need to further determine that out that the slope ground is uphill
Face is still descending ground.In order to allow ground detection model further to detect that target floor is upward slope ground or descending
Ground, during being trained to ground detection model, which includes the upward slope that floor type is upward slope type
Ground and floor type are the descending ground of descending type, wherein since the acquisition of image acquiring device is limited in scope, so,
It may not simultaneously include upward slope ground and descending ground in sample ground image.
In this way, technical staff can be according to the upward slope ground or descending ground in the sample image, in sample ordered three-dimensional
Put the position marked out in cloud upward slope type or descending type and upward slope or descending in sample ground image and relative to water
The inclination angle of plane, and by using the sample ordered three-dimensional point cloud as input, by the mark in the sample ordered three-dimensional point cloud
As a result deep neural network model parameter is trained as output.In this way, the ground detection model that training obtains can be examined
Measuring the slope ground is upward slope ground or descending ground.
In addition, it is contemplated that the location information on ground detection model output slope refers to position letter of the slope in ground image
Breath, and the location information in image can not intuitively reflect location information of the slope under world coordinate system very much.But
In practical application, mobile device is only determining that the slope, could be to traveling appearance after the location information under world coordinate system
State and mode are as accurately adjusting, therefore, in the disclosure, determine location information of the slope in ground image it
Afterwards, also need the location information location information being converted under world coordinate system.
Specifically, the ordered three-dimensional point cloud under the location information and world coordinate system according to slope in ground image, really
Determine location information of the slope under world coordinate system.Due to the ordered three-dimensional point cloud be aligned with ground image and the world
The coordinate of the every bit in ordered three-dimensional point cloud under coordinate system is its coordinate under world coordinate system, that is, real space
In coordinate, therefore, can based on the ordered three-dimensional point cloud under location information of the slope in ground image and world coordinate system,
Determine location information of the slope under world coordinate system.In this way, mobile device can be based on the slope under world coordinate system
Location information, adjust traveling posture and mode to ensure safety of advancing.
In one embodiment, if floor type characterization target floor is slope ground, prompt information is exported.
Under normal conditions, it easily causes danger when mobile device or visually impaired people advance at slope ground, in order into one
Step avoids dangerous generation, in the disclosure, when detecting target floor is slope ground, can also export prompt information,
Personnel in visually impaired people, automatic driving vehicle are prompted.Wherein, output prompting message can by voice, buzzer,
The various modes such as light are implemented.For example, voice prompting can be exported by microphone: having slope at 5 meters of front ground, please infuse
Meaning！
By adopting the above technical scheme, when detecting target floor is slope ground, prompt information can further be exported
User is prompted, so that user is predicted front state of ground in advance, and adjust traveling posture and mode in time, further to keep away
Exempt from dangerous generation.
Based on the same inventive concept, the disclosure also provides a kind of ground detection.Fig. 3 is according to an exemplary embodiment
A kind of block diagram of the ground detection shown, as shown in figure 3, the apparatus may include:
First obtains module 31, the ordered three-dimensional under the corresponding world coordinate system of ground image for obtaining target floor
Point cloud；
Input module 32, for obtaining using the ordered three-dimensional point cloud as the input of ground detection model trained in advance
To the corresponding ground detection of the target floor as a result, the ground detection result includes floor type, the floor type is used
It whether is the slope ground comprising slope in characterizing the target floor.
Optionally, the first acquisition module 31 may include:
Acquisition submodule, for obtaining the corresponding depth map of the ground image and the corresponding generation of the ground image
Attitude angle under boundary's coordinate system, the attitude angle include obtaining the image acquiring device of the ground image under world coordinate system
Angle；
Submodule is determined, for determining the corresponding world of the ground image according to the depth map and the attitude angle
Ordered three-dimensional point cloud under coordinate system.
Optionally, the determining submodule may include:
Normalization factor determines submodule, for determining normalization according to the depth map and preset Normalized Scale
The factor；
Depth map determines submodule, for the depth according to the depth map and the normalization factor, after determining normalization
Degree figure；
Point cloud determine submodule, for according to after the normalization depth map and the attitude angle, determine the ground
Ordered three-dimensional point cloud under the corresponding world coordinate system of image.
Optionally, if it is the slope ground, the ground detection result that the floor type, which characterizes the target floor,
Further include: the location information of inclination angle and the slope in the ground image of the slope with respect to the horizontal plane.
Optionally, described device can also include:
Second obtains module, for obtaining the sample ground image on sample ground；
Third obtains module, for obtaining the sample ordered three-dimensional under the corresponding world coordinate system of the sample ground image
Point cloud, the sample ordered three-dimensional point cloud be labeled with the sample floor type on the sample ground, the sample ground relative to
The sample position of the sample inclination angle of horizontal plane and the sample ground in the sample ground image；
Training module obtains described for being trained using the sample ordered three-dimensional point cloud as model training sample
Ground detection model.
Optionally, described device can also include:
Determining module, for according to the slope under the location information and the world coordinate system in the ground image
Ordered three-dimensional point cloud, determine location information of the slope under the world coordinate system.
Optionally, described device can also include:
Output module exports prompt if characterizing the target floor for the floor type is the slope ground
Information.
Those skilled in the art can be understood that, for convenience and simplicity of description, only with above-mentioned each function mould
The division progress of block can according to need and for example, in practical application by above-mentioned function distribution by different functional modules
It completes, i.e., the internal structure of device is divided into different functional modules, to complete all or part of the functions described above.
The specific work process of foregoing description functional module, can refer to corresponding processes in the foregoing method embodiment, no longer superfluous herein
It states.
Using above-mentioned apparatus, by the way that the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image is input to ground
Detection model, can be obtained the corresponding ground detection of the target floor as a result, without using artificial detection algorithm, so as to avoid
Artificial algorithm design difficulty is high, the problem that logic is complicated, robustness is bad.Also, due to according to the corresponding generation of ground image
Ordered three-dimensional point cloud under boundary's coordinate system detects ground, so, it can be to avoid disparity map in the related technology and depth map
In overexposure, the drawbacks of will appear more empty, biggish error under excessively dark, weak texture region.In addition, the ground detection model
It is that the neural network framework training based on existing RGB two dimensional image obtains, in this way, without constructing new neural network frame
The detection to ground can be realized using existing neural network framework in frame.
Fig. 4 is the block diagram of a kind of electronic equipment 400 shown according to an exemplary embodiment.Wherein, which can
To be robot device or the mobile helmet etc..As shown in figure 4, the electronic equipment 400 may include: processor 401, memory
402.The electronic equipment 400 can also include multimedia component 403, input/output (I/O) interface 404 and communication component
One or more of 405.
Wherein, processor 401 is used to control the integrated operation of the electronic equipment 400, to complete above-mentioned ground detection side
All or part of the steps in method.Memory 402 is for storing various types of data to support the behaviour in the electronic equipment 400
To make, these data for example may include the instruction of any application or method for operating on the electronic equipment 400, with
And the relevant data of application program, such as contact data, the message of transmitting-receiving, picture, audio, video etc..The memory 402
It can be realized by any kind of volatibility or non-volatile memory device or their combination, such as static random-access is deposited
Reservoir (Static Random Access Memory, abbreviation SRAM), electrically erasable programmable read-only memory
(Electrically Erasable Programmable Read-Only Memory, abbreviation EEPROM), erasable programmable
Read-only memory (Erasable Programmable Read-Only Memory, abbreviation EPROM), programmable read only memory
(Programmable Read-Only Memory, abbreviation PROM), and read-only memory (Read-Only Memory, referred to as
ROM), magnetic memory, flash memory, disk or CD.Multimedia component 403 may include screen and audio component.Wherein
Screen for example can be touch screen, and audio component is used for output and/or input audio signal.For example, audio component may include
One microphone, microphone is for receiving external audio signal.The received audio signal can be further stored in storage
Device 402 is sent by communication component 405.Audio component further includes at least one loudspeaker, is used for output audio signal.I/O
Interface 404 provides interface between processor 401 and other interface modules, other above-mentioned interface modules can be keyboard, mouse,
Button etc..These buttons can be virtual push button or entity button.Communication component 405 is for the electronic equipment 400 and other
Wired or wireless communication is carried out between equipment.Wireless communication, such as Wi-Fi, bluetooth, near-field communication (Near Field
Communication, abbreviation NFC), 2G, 3G, 4G, NB-IOT, eMTC or other 5G etc. or they one or more of
Combination, it is not limited here.Therefore the corresponding communication component 405 may include: Wi-Fi module, bluetooth module, NFC mould
Block etc..
In one exemplary embodiment, electronic equipment 400 can be by one or more application specific integrated circuit
(Application Specific Integrated Circuit, abbreviation ASIC), digital signal processor (Digital
Signal Processor, abbreviation DSP), digital signal processing appts (Digital Signal Processing Device,
Abbreviation DSPD), programmable logic device (Programmable Logic Device, abbreviation PLD), field programmable gate array
(Field Programmable Gate Array, abbreviation FPGA), controller, microcontroller, microprocessor or other electronics member
Part is realized, for executing above-mentioned ground detection method.
In a further exemplary embodiment, a kind of computer readable storage medium including program instruction is additionally provided, it should
The step of above-mentioned ground detection method is realized when program instruction is executed by processor.For example, the computer readable storage medium
It can be the above-mentioned memory 402 including program instruction, above procedure instruction can be executed by the processor 401 of electronic equipment 400
To complete above-mentioned ground detection method.
Fig. 5 is the block diagram of a kind of electronic equipment 500 shown according to another exemplary embodiment.For example, electronic equipment 500
It may be provided as a server.Referring to Fig. 5, electronic equipment 500 includes processor 522, and quantity can be one or more,
And memory 532, for storing the computer program that can be executed by processor 522.The computer journey stored in memory 532
Sequence may include it is one or more each correspond to one group of instruction module.In addition, processor 522 can be configured
To execute the computer program, to execute above-mentioned ground detection method.
In addition, electronic equipment 500 can also include power supply module 526 and communication component 550, which can be with
It is configured as executing the power management of electronic equipment 500, which, which can be configured as, realizes electronic equipment 500
Communication, for example, wired or wireless communication.In addition, the electronic equipment 500 can also include input/output (I/O) interface 558.Electricity
Sub- equipment 500 can be operated based on the operating system for being stored in memory 532, such as Windows ServerTM, Mac OS
XTM, UnixTM, LinuxTM etc..
In a further exemplary embodiment, a kind of computer readable storage medium including program instruction is additionally provided, it should
The step of above-mentioned ground detection method is realized when program instruction is executed by processor.For example, the computer readable storage medium
It can be the above-mentioned memory 532 including program instruction, above procedure instruction can be executed by the processor 522 of electronic equipment 500
To complete above-mentioned ground detection method.
In a further exemplary embodiment, a kind of computer program product is also provided, which includes energy
Enough computer programs executed by programmable device, which has is used for when being executed by the programmable device
Execute the code section of above-mentioned ground detection method.
The preferred embodiment of the disclosure is described in detail in conjunction with attached drawing above, still, the disclosure is not limited to above-mentioned reality
The detail in mode is applied, in the range of the technology design of the disclosure, a variety of letters can be carried out to the technical solution of the disclosure
Monotropic type, these simple variants belong to the protection scope of the disclosure.
It is further to note that specific technical features described in the above specific embodiments, in not lance
In the case where shield, it can be combined in any appropriate way.In order to avoid unnecessary repetition, the disclosure to it is various can
No further explanation will be given for the combination of energy.
In addition, any combination can also be carried out between a variety of different embodiments of the disclosure, as long as it is without prejudice to originally
Disclosed thought equally should be considered as disclosure disclosure of that.
Claims (10)
1. a kind of ground detection method characterized by comprising
Obtain the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image of target floor；
Using the ordered three-dimensional point cloud as the input of ground detection model trained in advance, it is corresponding to obtain the target floor
As a result, the ground detection result includes floor type, the floor type is ground detection for characterizing the target floor
No is the slope ground comprising slope.
2. the method according to claim 1, wherein the corresponding world of ground image for obtaining target floor
Ordered three-dimensional point cloud under coordinate system, comprising:
The attitude angle under the corresponding depth map of the ground image and the corresponding world coordinate system of the ground image is obtained,
The attitude angle includes the angle for obtaining the image acquiring device of the ground image under world coordinate system；
According to the depth map and the attitude angle, the ordered three-dimensional point under the corresponding world coordinate system of the ground image is determined
Cloud.
3. according to the method described in claim 2, it is characterized in that, described according to the depth map and the attitude angle, determination
Ordered three-dimensional point cloud under the corresponding world coordinate system of the ground image, comprising:
According to the depth map and preset Normalized Scale, normalization factor is determined；
Depth map according to the depth map and the normalization factor, after determining normalization；
According to after the normalization depth map and the attitude angle, determine under the corresponding world coordinate system of the ground image
Ordered three-dimensional point cloud.
4. the method according to claim 1, wherein if it is described that the floor type, which characterizes the target floor,
Slope ground, the ground detection result further include: slope inclination angle with respect to the horizontal plane and the slope are described
Location information in the image of face.
5. according to the method described in claim 4, it is characterized in that, the ground detection model is trained in the following manner
It arrives:
Obtain the sample ground image on sample ground；
Obtain the sample ordered three-dimensional point cloud under the corresponding world coordinate system of the sample ground image, the sample ordered three-dimensional
Point cloud is labeled with the sample floor type on the sample ground, the sample ground sample inclination angle with respect to the horizontal plane and institute
State sample position of the sample ground in the sample ground image；
It is trained using the sample ordered three-dimensional point cloud as model training sample, obtains the ground detection model.
6. according to the method described in claim 4, it is characterized in that, the method also includes:
According to ordered three-dimensional point cloud of the slope under the location information and the world coordinate system in the ground image, really
Fixed location information of the slope under the world coordinate system.
7. method according to any one of claim 1 to 6, which is characterized in that the method also includes:
If it is the slope ground that the floor type, which characterizes the target floor, prompt information is exported.
8. a kind of ground detection characterized by comprising
First obtains module, the ordered three-dimensional point cloud under the corresponding world coordinate system of ground image for obtaining target floor；
Input module, for obtaining described using the ordered three-dimensional point cloud as the input of ground detection model trained in advance
The corresponding ground detection of target floor is as a result, the ground detection result includes floor type, and the floor type is for characterizing
Whether the target floor is the slope ground comprising slope.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The step of any one of claims 1 to 7 the method is realized when row.
10. a kind of electronic equipment characterized by comprising
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize any one of claims 1 to 7 institute
The step of stating method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910563488.1A CN110378246A (en) | 2019-06-26 | 2019-06-26 | Ground detection method, apparatus, computer readable storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910563488.1A CN110378246A (en) | 2019-06-26 | 2019-06-26 | Ground detection method, apparatus, computer readable storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110378246A true CN110378246A (en) | 2019-10-25 |
Family
ID=68250945
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910563488.1A Pending CN110378246A (en) | 2019-06-26 | 2019-06-26 | Ground detection method, apparatus, computer readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378246A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111123278A (en) * | 2019-12-30 | 2020-05-08 | 科沃斯机器人股份有限公司 | Partitioning method, device and storage medium |
| CN111586299A (en) * | 2020-05-09 | 2020-08-25 | 北京华捷艾米科技有限公司 | Image processing method and related equipment |
| CN111754636A (en) * | 2020-06-26 | 2020-10-09 | 北京百度网讯科技有限公司 | Three-dimensional point cloud processing method, three-dimensional point cloud processing model training method and device |
| CN112580428A (en) * | 2020-11-09 | 2021-03-30 | 义乌市输变电工程有限公司设计分公司 | Power distribution network design method and device |
| CN113848884A (en) * | 2021-09-07 | 2021-12-28 | 华侨大学 | Unmanned engineering machinery decision method based on feature fusion and space-time constraint |
| WO2023015914A1 (en) * | 2021-08-12 | 2023-02-16 | 达闼机器人股份有限公司 | Method and apparatus for obtaining gravity direction of image, electronic device, and storage medium |
| CN116385528A (en) * | 2023-03-28 | 2023-07-04 | 小米汽车科技有限公司 | Method and device for generating annotation information, electronic equipment, vehicle and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1286178A2 (en) * | 2001-08-23 | 2003-02-26 | IBEO Automobile Sensor GmbH | Method for optical ground detection |
| US20130096886A1 (en) * | 2010-03-31 | 2013-04-18 | Borys Vorobyov | System and Method for Extracting Features from Data Having Spatial Coordinates |
| CN103287582A (en) * | 2012-02-15 | 2013-09-11 | 空中客车运营简化股份公司 | Detection of aircraft descent anomaly |
| US20160330434A1 (en) * | 2015-05-08 | 2016-11-10 | Lite-On Electronics (Guangzhou) Limited | Control method of a depth camera |
| CN108245385A (en) * | 2018-01-16 | 2018-07-06 | 曹醒龙 | A kind of device for helping visually impaired people's trip |
| CN108363995A (en) * | 2018-03-19 | 2018-08-03 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating data |
| CN108614274A (en) * | 2018-05-22 | 2018-10-02 | 广州市极臻智能科技有限公司 | Staggered form crossover track distance measurement method based on multi-rotor unmanned aerial vehicle and device |
| CN108885791A (en) * | 2018-07-06 | 2018-11-23 | 深圳前海达闼云端智能科技有限公司 | ground detection method, related device and computer readable storage medium |
| CN109002039A (en) * | 2018-07-06 | 2018-12-14 | 达闼科技（北京）有限公司 | Avoidance based reminding method, relevant apparatus and computer readable storage medium |
| CN109074668A (en) * | 2018-08-02 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method for path navigation, relevant apparatus and computer readable storage medium |
| US10186049B1 (en) * | 2017-03-06 | 2019-01-22 | URC Ventures, Inc. | Determining changes in object structure over time using mobile device images |
| US20190188460A1 (en) * | 2017-12-18 | 2019-06-20 | Arcsoft (Hangzhou) Multimedia Technology Co., Ltd. | Method and device for use in hand gesture recognition |
-
2019
- 2019-06-26 CN CN201910563488.1A patent/CN110378246A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1286178A2 (en) * | 2001-08-23 | 2003-02-26 | IBEO Automobile Sensor GmbH | Method for optical ground detection |
| US20130096886A1 (en) * | 2010-03-31 | 2013-04-18 | Borys Vorobyov | System and Method for Extracting Features from Data Having Spatial Coordinates |
| CN103287582A (en) * | 2012-02-15 | 2013-09-11 | 空中客车运营简化股份公司 | Detection of aircraft descent anomaly |
| US20160330434A1 (en) * | 2015-05-08 | 2016-11-10 | Lite-On Electronics (Guangzhou) Limited | Control method of a depth camera |
| US10186049B1 (en) * | 2017-03-06 | 2019-01-22 | URC Ventures, Inc. | Determining changes in object structure over time using mobile device images |
| US20190188460A1 (en) * | 2017-12-18 | 2019-06-20 | Arcsoft (Hangzhou) Multimedia Technology Co., Ltd. | Method and device for use in hand gesture recognition |
| CN108245385A (en) * | 2018-01-16 | 2018-07-06 | 曹醒龙 | A kind of device for helping visually impaired people's trip |
| CN108363995A (en) * | 2018-03-19 | 2018-08-03 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating data |
| CN108614274A (en) * | 2018-05-22 | 2018-10-02 | 广州市极臻智能科技有限公司 | Staggered form crossover track distance measurement method based on multi-rotor unmanned aerial vehicle and device |
| CN108885791A (en) * | 2018-07-06 | 2018-11-23 | 深圳前海达闼云端智能科技有限公司 | ground detection method, related device and computer readable storage medium |
| CN109002039A (en) * | 2018-07-06 | 2018-12-14 | 达闼科技（北京）有限公司 | Avoidance based reminding method, relevant apparatus and computer readable storage medium |
| CN109074668A (en) * | 2018-08-02 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method for path navigation, relevant apparatus and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 朱株: "基于三维数据面向无人车导航的非结构化场景理解", 《中国博士学位论文全文数据库（工程科技Ⅱ辑）》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111123278A (en) * | 2019-12-30 | 2020-05-08 | 科沃斯机器人股份有限公司 | Partitioning method, device and storage medium |
| CN111123278B (en) * | 2019-12-30 | 2022-07-12 | 科沃斯机器人股份有限公司 | Partitioning method, partitioning equipment and storage medium |
| CN111586299A (en) * | 2020-05-09 | 2020-08-25 | 北京华捷艾米科技有限公司 | Image processing method and related equipment |
| CN111586299B (en) * | 2020-05-09 | 2021-10-19 | 北京华捷艾米科技有限公司 | Image processing method and related equipment |
| CN111754636A (en) * | 2020-06-26 | 2020-10-09 | 北京百度网讯科技有限公司 | Three-dimensional point cloud processing method, three-dimensional point cloud processing model training method and device |
| CN111754636B (en) * | 2020-06-26 | 2023-07-25 | 北京百度网讯科技有限公司 | Three-dimensional point cloud processing method, three-dimensional point cloud processing model training method and device |
| CN112580428A (en) * | 2020-11-09 | 2021-03-30 | 义乌市输变电工程有限公司设计分公司 | Power distribution network design method and device |
| WO2023015914A1 (en) * | 2021-08-12 | 2023-02-16 | 达闼机器人股份有限公司 | Method and apparatus for obtaining gravity direction of image, electronic device, and storage medium |
| CN113848884A (en) * | 2021-09-07 | 2021-12-28 | 华侨大学 | Unmanned engineering machinery decision method based on feature fusion and space-time constraint |
| CN113848884B (en) * | 2021-09-07 | 2023-05-05 | 华侨大学 | Unmanned engineering machinery decision method based on feature fusion and space-time constraint |
| CN116385528A (en) * | 2023-03-28 | 2023-07-04 | 小米汽车科技有限公司 | Method and device for generating annotation information, electronic equipment, vehicle and storage medium |
| CN116385528B (en) * | 2023-03-28 | 2024-04-30 | 小米汽车科技有限公司 | Method and device for generating annotation information, electronic equipment, vehicle and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110378246A (en) | Ground detection method, apparatus, computer readable storage medium and electronic equipment | |
| US11915501B2 (en) | Object detection method and apparatus, electronic device, and storage medium | |
| CN106407981B (en) | A kind of licence plate recognition method, apparatus and system | |
| CN109657553B (en) | Student classroom attention detection method | |
| CN105023010B (en) | A kind of human face in-vivo detection method and system | |
| CN101068344B (en) | Object detection apparatus | |
| CN105447853B (en) | Flight instruments, flight control system and method | |
| CN103765870A (en) | Image processing apparatus, projector and projector system including image processing apparatus, image processing method | |
| CN103473542B (en) | Multi-clue fused target tracking method | |
| CN110378965A (en) | Determine the method, apparatus, equipment and storage medium of coordinate system conversion parameter | |
| CN105096307B (en) | The method of detection object in paired stereo-picture | |
| EP2458405A1 (en) | Graphics-aided remote position measurement with handheld geodesic device | |
| CN106125994B (en) | Coordinate matching method and the control method and terminal for using the coordinate matching method | |
| US11315264B2 (en) | Laser sensor-based map generation | |
| CN105654067A (en) | Vehicle detection method and device | |
| CN109002039A (en) | Avoidance based reminding method, relevant apparatus and computer readable storage medium | |
| CN108885791A (en) | ground detection method, related device and computer readable storage medium | |
| CN106361345A (en) | System and method for measuring height of human body in video image based on camera calibration | |
| CN105931263A (en) | Target tracking method and electronic equipment | |
| CN104964708B (en) | A kind of road surface pit detection method based on vehicle-mounted binocular vision | |
| CN108474658A (en) | Ground Morphology observation method and system, unmanned plane landing method and unmanned plane | |
| KR102297202B1 (en) | Display device for vehicle | |
| CN109253722A (en) | Merge monocular range-measurement system, method, equipment and the storage medium of semantic segmentation | |
| CN106570899A (en) | Target object detection method and device | |
| CN106323242A (en) | Space structure detection method and device for unmanned aerial vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210303 Address after: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant after: Dalu Robot Co.,Ltd. Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant before: Shenzhen Qianhaida Yunyun Intelligent Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| CB02 | Change of applicant information |
Address after: 201111 Building 8, No. 207, Zhongqing Road, Minhang District, Shanghai Applicant after: Dayu robot Co.,Ltd. Address before: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant before: Dalu Robot Co.,Ltd. |
|
| CB02 | Change of applicant information |