CN110990594A - Robot space cognition method and system based on natural language interaction - Google Patents
Robot space cognition method and system based on natural language interaction Download PDFInfo
- Publication number
- CN110990594A CN110990594A CN201911207208.XA CN201911207208A CN110990594A CN 110990594 A CN110990594 A CN 110990594A CN 201911207208 A CN201911207208 A CN 201911207208A CN 110990594 A CN110990594 A CN 110990594A
- Authority
- CN
- China
- Prior art keywords
- distance
- point
- target
- target object
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/387—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Library & Information Science (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention discloses a robot space cognition method and system based on natural language interaction, which comprises the following steps: establishing a spatial information corpus based on natural language expression, comprising: a target attribute feature description corpus and a target location feature description corpus; converting the target attribute feature description corpus and the target position feature description corpus into a keyword array according to a preset grammar rule; judging the calculation relationship between the object type and the spatial position of the target object and the reference object according to the related characteristics of the object contained in the keyword array, wherein the calculation relationship of the spatial position comprises at least one of the following relationships: the direction relation of the target object relative to the reference object, the distance relation of the target object relative to the reference object and the topological relation of the target object relative to at least two reference objects; and determining the coordinate range of the target object according to the calculation relationship between the categories and the spatial positions of the target object and the reference object so as to search for the target object in the following process. The invention can reduce the interaction frequency between the human and the robot and improve the human-computer interaction efficiency.
Description
Technical Field
The invention relates to the technical field of human-computer interaction, in particular to a robot space cognition method and system based on natural language interaction.
Background
Under unstructured environments such as planet detection and field rescue, the robot cannot understand all environmental information, and the difficulty and efficiency of independently completing tasks still have great problems. The capability advantages of people in the aspects of comprehensive perception, prediction judgment, spatial reasoning and the like can well make up for the deficiency, and man-machine cooperation is an effective method for executing tasks. The man-machine interaction mode is one of key points in the man-machine cooperation process, and the man-machine interaction mode which is natural and friendly can effectively improve the interaction level. The natural language is one of natural interaction modes, is unlimited, does not need to distort natural thinking and behavior modes to adapt to the requirements of the robot, has low requirements on environment and equipment, is suitable for unstructured environment, and is widely applied to the field of mobile robots.
The process of completing tasks by human-computer cooperation relates to the processing of spatial information, namely spatial cognition. Because the space cognition mechanism between the robots is greatly different, the robots are difficult to understand the space information expressed based on the natural language and can only receive the control instruction with the unidirectional structure, and the operation efficiency is greatly influenced by frequent and low-speed interaction. To solve this problem, the emphasis is on how the robot can understand the cognitive expression of the human on the spatial information. In the prior art, the simulation of the human cognition process is realized based on a cognitive theory framework, namely, a space reference framework selection and space inference characteristic in human language command and communication is obtained through experiments, and a space cognition and inference module is tried to be constructed for a robot. In the field of space information interaction oriented to man-machine cooperation, a robot space cognition method meeting natural language interaction needs to be urgently needed.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to solve the technical problems that the space cognition mechanism between the robots is greatly different, the robots are difficult to understand the space information expressed based on the natural language, only can receive unidirectional structured control instructions, and frequent and low-speed interaction greatly influences the operation efficiency.
In order to achieve the above object, in a first aspect, the present invention provides a robot space recognition method based on natural language interaction, including the following steps:
establishing a spatial information corpus based on natural language expression, comprising: a target attribute feature description corpus and a target location feature description corpus;
converting the target attribute feature description corpus and the target position feature description corpus into a keyword array according to a preset grammar rule; the keywords include: the target object name, the reference object name, the direction relation and the distance relation;
according to the related characteristics of the objects contained in the keyword array, judging the categories of the target object and the reference object and the spatial position calculation relationship, wherein the categories comprise a point object and a planar object, and the spatial position calculation relationship comprises at least one of the following relationships: the direction relation of the target object relative to the reference object, the distance relation of the target object relative to the reference object and the topological relation of the target object relative to at least two reference objects;
and determining the coordinate range of the target object according to the calculation relationship among the target object, the category of the reference object and the space position so as to search for the target object in the following process.
Optionally, the category to which the object belongs is determined by:
if the object of the category to be judged is an independent object, abstracting the object into a point object or a planar object without influencing the spatial position expression of the object or other objects except the object of the category to be judged, and regarding the object as a point object;
if the area ratio of the object of the category to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted to be a point object and the spatial position expression of the object or other objects except the object of the category to be judged is influenced.
Optionally, according to the directional relationship of the object with respect to the reference object, the coordinate range of the object is obtained by the following steps:
when the reference object is a point object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object at the origin of a coordinate system, and for any point-shaped target object in the space, obtaining a coordinate position set of the point-shaped target object relative to the point-shaped reference object in different directions according to a plurality of preset straight line constraints;
when the reference object is a planar reference object, determining the planar reference object and a minimum circumscribed rectangle thereof by using a minimum boundary rectangle model, and taking straight lines where four rectangle sides of the minimum circumscribed rectangle are located as boundary lines in all directions; determining a coordinate position set of the point-shaped object relative to the planar reference object in different directions according to the boundary lines in all directions;
if two reference objects exist, the coordinate position ranges of the target object are determined respectively only according to different reference object position descriptions, and then the intersection of the two ranges is obtained.
Optionally, according to a distance relationship between the target object and the reference object, the distance relationship includes: quantitative, qualitative, or temporal distances; solving the coordinate range of the object by the following steps:
when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range area;
when the distance relation is a qualitative distance, presetting different distance thresholds for distances of different granularity levels, and if the reference object is a point-shaped reference object, setting the distance of the point-shaped target object from the point-shaped reference object as a qualitative distance range area;
when the distance relation is time distance, converting the time distance into quantitative distance, and then determining the coordinate range of the point-like reference object;
when the distance relationship between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object need to be respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
Optionally, according to the distance relationship and the direction relationship of the target object relative to the reference object, the coordinate range of the target object is solved through the following steps:
and solving the coordinate range of the target object according to two constraint conditions of the distance relation and the direction relation of the target object relative to the reference object, and finally solving the intersection of the two coordinate ranges to determine the final coordinate range of the target object.
Optionally, according to the topological relation of the object with respect to the at least two reference objects, the coordinate range of the object is solved by the following steps:
if the topological relation is that the target object is between the two reference objects:
when the two reference objects are both point reference objects, the target object is in the range of the line segment formed by connecting the two reference objects and the area with the distance line segment as the preset distance;
when the two reference objects are planar reference objects, the range of the target object is determined according to the rectangular edge of the minimum circumscribed rectangle of the two planar reference objects;
when the two reference objects are the point-like reference object and the planar reference object, the range of the target object is determined according to the coordinates of the point-like reference object and the rectangular side of the rectangular minimum circumscribed rectangle of the planar reference object.
In a second aspect, the present invention provides a robot spatial recognition system based on natural language interaction, including:
the corpus establishing unit is used for establishing a spatial information corpus based on natural language expression, and comprises the following steps: a target attribute feature description corpus and a target location feature description corpus;
a keyword determining unit, configured to convert the target attribute feature description corpus and the target location feature description corpus into a keyword array according to a preset grammar rule; the keywords include: the target object name, the reference object name, the direction relation and the distance relation;
a feature determination unit, configured to determine, according to the object-related features included in the keyword array, a category and a spatial position calculation relationship to which the target object and the reference object belong, where the category includes a point object and a planar object, and the spatial position calculation relationship includes at least one of the following relationships: the direction relation of the target object relative to the reference object, the distance relation of the target object relative to the reference object and the topological relation of the target object relative to at least two reference objects;
and the target object coordinate determining unit is used for determining the coordinate range of the target object according to the target object, the category of the reference object and the spatial position calculation relationship so as to search for the target object subsequently.
Optionally, the keyword determination unit determines the category to which the object belongs by: if the object of the category to be judged is an independent object, abstracting the object into a point object or a planar object without influencing the spatial position expression of the object or other objects except the object of the category to be judged, and regarding the object as a point object; if the area ratio of the object of the category to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted to be a point object and the spatial position expression of the object or other objects except the object of the category to be judged is influenced.
Optionally, according to the directional relationship of the object with respect to the reference object, the object coordinate determining unit solves the coordinate range of the object by: when the reference object is a point object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object at the origin of a coordinate system, and for any point-shaped target object in the space, obtaining a coordinate position set of the point-shaped target object relative to the point-shaped reference object in different directions according to a plurality of preset straight line constraints; when the reference object is a planar reference object, determining the planar reference object and a minimum circumscribed rectangle thereof by using a minimum boundary rectangle model, and taking straight lines where four rectangle sides of the minimum circumscribed rectangle are located as boundary lines in all directions; determining a coordinate position set of the point-shaped object relative to the planar reference object in different directions according to the boundary lines in all directions; if two reference objects exist, the coordinate position ranges of the target object are determined respectively only according to different reference object position descriptions, and then the intersection of the two ranges is obtained.
Optionally, according to a distance relationship between the target object and the reference object, the distance relationship includes: quantitative, qualitative, or temporal distances; the object coordinate determination unit solves the coordinate range of the object by the following steps: when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range area; when the distance relation is a qualitative distance, presetting different distance thresholds for distances of different granularity levels, and if the reference object is a point-shaped reference object, setting the distance of the point-shaped target object from the point-shaped reference object as a qualitative distance range area; when the distance relation is time distance, converting the time distance into quantitative distance, and then determining the coordinate range of the point-like reference object; when the distance relationship between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object need to be respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
compared with a space cognition mode that a traditional robot mainly receives a unidirectional structured control instruction, the robot space cognition method and system based on natural language interaction increase the cognition of the robot to natural language, find out keywords in the natural language instruction and determine the coordinate range of a target object from the keywords, are convenient for the robot to perceive the natural language, can reduce the interaction frequency between the robot and the robot, and improve the man-machine interaction efficiency.
Compared with the traditional robot space cognition mode in which people mainly send structured control instructions and need to learn a large number of expression rules, the robot space cognition method and system based on natural language interaction provided by the invention only need to use a natural language expression mode to express space information, and the cognition load is reduced.
Drawings
FIG. 1 is a schematic flow chart of a robot space cognition method based on natural language interaction according to the present invention;
FIG. 2 is a schematic diagram of an eight-directional cone model provided by the present invention;
FIG. 3 is a schematic diagram of a Minimum Bounding Rectangle (MBR) model provided by the present invention;
FIG. 4 is a schematic diagram of coordinate system transformation provided by the present invention;
FIG. 5 is a schematic diagram illustrating a distance relationship provided by the present invention;
FIG. 6 is a schematic diagram of a point-like reference object description only described in a topological relation provided by the present invention;
FIG. 7 is a schematic diagram of a planar reference object described only by a topological relation provided by the present invention;
FIG. 8 is a schematic diagram of the description of a "point-like + planar" reference object only described by a topological relation provided by the present invention;
fig. 9 is a robot space recognition system architecture diagram based on natural language interaction provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
In order to solve the defects of the prior art, namely the problem that the robot is difficult to recognize and understand the spatial information expressed based on the natural language in the complex task scene, the invention provides a robot spatial recognition method meeting natural language interaction, and the robot can understand the complex spatial information expressed based on the natural language, such as target description, positioning and the like. The specific technical route is as follows:
the robot space cognition method comprises a natural language processing module and a space cognition module, and is divided into five main steps.
(1) And establishing a corresponding spatial information expression corpus facing to a task scene, wherein the corpus comprises target object attribute feature description and target position description corpora.
(2) And the natural language processing module converts the spatial information expressed based on the natural language into related keyword arrays according to grammar rules and the corpus in the step 1, wherein the keyword arrays comprise the name of the target object, the attribute of each dimension of the target object, the type of the target object, the name of the reference object and the like.
(3) And the space cognition module stores knowledge related to the object characteristics according to the keyword array and judges the category and the spatial position calculation type of the object. The object belongs to the category comprising a point object and a planar object, and the spatial position calculation type comprises a single (two) reference object + direction relation, a single (two) reference object + distance relation, a single (two) reference object + direction relation + distance relation and a two reference object + only topological relation.
(4) And the space cognition module calculates the space position according to the space relations of different types.
(5) And the space cognition module is used for completing the matching of the target object characteristics and the target object coordinates.
As shown in fig. 1, a technical route diagram of the robot spatial awareness method is shown. The system is divided into two main parts, namely a natural language processing module and a space cognition module. The natural language processing module comprises a space information corpus, processes space information expressed by natural language and analyzes corresponding keywords. The space cognition module completes the calculation reasoning function of the space position. After receiving the keywords, storing knowledge related to the object characteristics according to the keyword array, judging the category of the reference object and the spatial position calculation type related to the target object, performing reasoning calculation of the spatial position according to different types, and finally completing matching of the target object characteristics and the target object coordinates. The specific contents are detailed as follows:
1. corpus creation and natural language parsing
According to the task scene, after a large amount of linguistic data expressed by spatial information are collected through experiments, a spatial information corpus is formed, corresponding natural language expression rules are obtained, and important keywords are extracted from the linguistic data, wherein the linguistic data comprises the following steps: target object name, reference object name, orientation relationship, distance relationship, etc. On the basis, the natural language can be analyzed by compiling a corpus covering the task process. For example, the natural language expresses that the red flag is 5 meters to the left of the stone. "resolvable to" target name based on expression rules: red flag, reference: stone, azimuth relation: left, distance relationship: 5 m ", table 1 shows part of the grammar rules.
Table 1 grammar rules example table
In table 1, "{ }" denotes a keyword, "[ ]" denotes an assist word, and "|" denotes a relationship of or.
2. And judging the object type and the spatial relationship calculation type.
Since the description of the position of a human to a target object is related to the nature and characteristics of the object, the object subjected to a spatial cognitive task is divided into two types, namely a point-like object and a planar object, by referring to related concepts in geography. If the object is an independent object, the area ratio is of unimportant attribute, namely the object is abstracted to be a point object or a planar object without influencing the spatial position expression of the object or other objects, and the object can be regarded as the point object; if the area ratio of the object is large, the spatial position expression of the object or other objects is greatly influenced when the object is abstracted to be a point object, and the object is regarded as a planar object.
In view of the spatial recognition scenario, the target object to be located is generally regarded as a point object, and the reference object may be a point or a planar object. The judgment of the object belonging category mainly depends on the prior knowledge base of the robot, and the object belonging category is obtained by matching the name of the target object or the reference object in the keyword array with the knowledge base.
The position description of the target object is a representation of the spatial relationship of different objects, namely, the object position is commonly characterized by the target object, the reference object and the spatial relationship.
The reference can be divided into a single reference and two references. The spatial relationship comprises a topological relationship, a directional relationship and a distance relationship.
The topological relation refers to the relation that each space element is connected, separated, adjacent and the like without considering the specific position. When a person uses a natural language to describe the position of an object, the description of the phase-separation relationship is mainly involved, and the topological relationship such as intersection, inclusion and the like is basically absent, and the phase-separation relationship is generally considered by default.
Directional relationships refer to the position of one object in space relative to one or more other objects. In describing the directional relationship, at least three elements are generally required, namely the target object, the reference object, and the reference frame used. The directional relationship can be divided into an absolute directional relationship and a relative directional relationship according to the used reference frame. The absolute directional relationship means that a world coordinate system is used when the directional relationship description is made, and the relative directional relationship means that a relative coordinate system is used when the directional relationship description is made.
The distance relationship reflects the geometric proximity between objects in different spaces, and the daily life uses a quantitative distance, i.e. a distance described by a numerical value, which is generally based on an artificially set measurement system.
Qualitative analysis can be divided into two categories, one is described by using a degree adverb + quantitative distance of about 5 m; and the other is described using only the very similar adverbs.
The time distance is expressed in the form "moving pattern + time".
Combining the above results, the calculation types of the spatial position are divided into the following categories: (1) single (two) reference + direction relationship; (2) single (two) reference + distance relationship; (3) single (two) reference objects + direction relation + distance relation; (4) two references + topological relationship only.
3. Calculation of spatial position
(1) Single (two) reference object + direction relation
The directional relationship can be divided into an absolute directional relationship and a relative directional relationship. On the premise of using the absolute direction relation: when the reference object is a point-like reference object, an eight-direction conical model is generally adopted, and as shown in fig. 2, the model divides the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees.
In order to accurately describe the directional relationship of the space object, a coordinate system as shown in fig. 2 is established. Assuming that the point-like reference object A is located at the origin O of the coordinate system, then for any point-like object target B in space, according to L1、L2、L3、L4And (3) linear constraint, namely obtaining a coordinate set of the point-shaped reference object A in different directions, as shown in the table 2.
TABLE 2 schematic table for calculating spatial relationship of point-like reference object
When the reference object is a planar reference object, a Minimum Bounding Rectangle (MBR) model is used, as shown in fig. 3. In fig. 3, the hatched portion is a planar reference object, which can be referred to as a, and the rectangle abcd is the smallest circumscribed rectangle, so that the straight line side where the four rectangular sides ab, ac, bc, and bd are located can be used as the boundary in each direction.
A coordinate system as shown in fig. 3 is established. Four vertices of a minimum bounding rectangle abcd of the planar reference object a are a (x)1，y1)，b(x1，y2)，c(x2，y2)，d(x2，y1) For any point object in space, target B, according to L1、L2、L3、L4The constraint of the linear equation can obtain the coordinate set of the planar reference object A in different directions,as shown in table 3, Dir () in table 3 represents its orientation. If the relative direction relationship is used, it needs to be converted into the absolute direction relationship. I.e. the conversion from the relative coordinate system to the world coordinate system is completed.
TABLE 3 schematic space calculation table for planar reference object
And establishing a coordinate system by taking the starting point of the robot as the origin of a world coordinate system, taking the positive east direction as the positive direction of the x axis and taking the positive north direction as the positive direction of the y axis. Suppose that the position of the robot at time t is (x)t，yt) The deflection angle is θ (relative to the positive x-axis), as shown in FIG. 4.
The position of the object P is the right front of the robot, and the coordinate is P' (x) in a coordinate system centered on the robotp'，yp') now converted to coordinates in the world coordinate system, assumed to be P (x)p，yp) The conversion formula of the two can be obtained by derivation according to the geometric relationship as follows:
similarly, when the origin coordinates (a, b) of the relative coordinate system, the deflection angle θ of the relative coordinate system with respect to the world coordinate system, and the position coordinate (x) of the point P in the relative coordinate system are determinedp'，yp') can be converted to obtain the coordinate P (x) of the point P in the world coordinate systemp，yp) The method comprises the following steps:
if two reference objects exist, the coordinate positions of the objects are determined respectively only according to different reference object position descriptions, and then the intersection of the two ranges is obtained.
(2) Single (two) reference object + distance relation
The distance relationship includes quantitative distance, qualitative distance and timeDistance. Since the distance relationship is more used to describe the point-like reference object, the distance relationship is considered to be the point-like reference object in a simplified manner. A quantitative distance is a distance described by a numerical value, typically based on an artificially set metrology system. However, since the cognitive abilities of people to the space are not completely the same, and the description of the partial quantitative distance has a large deviation, an error parameter d needs to be introduced when the position of the distance relationship is calculated. In the description of natural language, when the distance between the point-shaped target object A and the point-shaped reference object B is D, the actual distance D is1D ± D. According to the related conclusions of the prior art, it can be approximately considered that:
d＝k×D (3)
in the formula (3), k is an error proportionality coefficient.
As shown in fig. 5, a is a point-like reference object, and if a point-like object B is considered to be at a distance D from a, the gray area range is the actual area of B.
Qualitative distances can be broadly divided into those described using the "degree adverb + quantitative distance" and those described using only the degree adverb, such as "where the ball is in close proximity to the cart".
For the first case, a quantitative distance calculation mode can be directly adopted, and the error condition is considered when the quantitative distance is used for position calculation, so that the error does not need to be calculated; for the second case, the concept of qualitative distance description framework is adopted, and distance relation division can be performed on different granularity levels according to different research tasks.
The four qualitative distance relationship granularity levels, namely, very close, near, far and very far, are introduced, and different qualitative distance relationships are subjected to quantization processing and set as quantitative distances, as shown in table 4:
TABLE 4 qualitative distance relationship correspondence table
The time distance is converted into a quantitative distance for calculation. If the time distance is L, the speed is v, and the consumed time is t, then the distance relationship is:
L＝vt (4)
the moving speed v of the introduced robot is 2m/s to participate in calculation, and the time-distance relation is quantified.
When the distance relationship between two reference objects is used for describing the position of the target object, the coordinate ranges of the target object need to be respectively determined according to different reference object distance descriptions, and the intersection is obtained by the two reference object distance descriptions.
(3) Single (two) reference object + direction relation + distance relation
In the foregoing model, the relative direction relationship may be converted into an absolute direction relationship, and the qualitative distance relationship and the time-distance relationship may be converted into a quantitative distance relationship, so that the different types of "direction relationship + distance relationship" are finally converted into "absolute direction relationship + quantitative distance relationship" for position calculation.
If the target object M is located 5 meters east of the dotted reference object a (0, 0), the coordinate constraint of M is:
wherein x and y represent the abscissa and ordinate values of the target M, respectively.
When the direction relation and the distance relation of the two reference objects are used for description, the direction relation and the distance relation are respectively converted into an absolute direction relation and a quantitative distance relation, the respective constraint conditions of the two reference objects are calculated, and the intersection of the two reference objects is obtained.
(4) Two reference objects + topological relation only
There are other spatial relationships that are not categorized in the direction and distance relationships, and the spatial relationship expression is described using only the topological relationship, such as "ball between you and cart", as discussed below.
If the two reference objects are both point-like reference objects, the two point-like reference objects are respectively set as a and B, as shown in fig. 6, the line segment AB is taken as a central axis, the possible positions of the target object are around the line segment AB, and in order to quantitatively calculate the possible positions, an error parameter d is introduced, namely the target object is in an area with the distance d from the line segment AB.
Because the slope k of the straight line AB is different, the expression of the formed region is also different, as shown in fig. 6. And respectively calculating the M coordinate range set of the target object under different slopes.
In FIG. 6 (a), there is A (x)A，yA)，B(xA，yB). The M coordinate ranges are:
in FIG. 6 (b), there is A (x)A，yA)，B(xB，yA). The M coordinate ranges are:
in FIG. 6 (c), there is A (x)A，yA)，B(xB，yB). The M coordinate ranges are:
wherein x isM，yMRespectively, the abscissa and ordinate values of the target M, and θ represents the deflection angle of the robot with respect to the world coordinate system.
If both the two reference objects are planar reference objects, as shown in fig. 7, the target object is M, A, B are the two planar reference objects, the rectangles abcd and efgh are minimum circumscribed rectangles of A, B, respectively, and the gray area in the figure is the position area of the target object M.
The location areas of M in different cases are calculated separately.
In FIG. 7 (a), there is a (x)a，ya)，b(xb，ya)，c(xb，yc)，d(xa，yc)，e(xe，ye)，f(xf，yf)，g(xg，yg)，h(xh，yh). M coordinate rangeThe method comprises the following steps:
in FIG. 7 (b), there is a (x)a，ya)，b(xb，ya)，c(xb，yc)，d(xa，yc)，e(xe，ye)，f(xf，yf)，g(xg，yg)，h(xh，yh). The M coordinate ranges are:
when the two reference objects are the dot reference object and the planar reference object, respectively, as shown in fig. 8, the target object M has the dot reference object a, the planar reference object B, and the rectangle abcd is the smallest circumscribed rectangle of the B. The two cases can be divided into two cases in fig. 8 according to the relative direction of a and B, and the gray area in the figure is M range.
With a (x)a，ya)，b(xb，ya)，c(xb，yc)，d(xa，yc)，A(xA，yA) The position areas of the target M in different cases are calculated, respectively.
In fig. 8 (a), there are M coordinate ranges:
in fig. 8 (b), there are M coordinate ranges:
4. object feature matching
And the spatial cognition module performs matching synthesis on the name and the attribute characteristics of the target object and the calculated object coordinate constraint range. Assuming that the set of attributes of object A is N, then there are:
n ═ name, color, size, shape, coordinate }
Considering the requirement of the robot for path planning, selecting a point with the shortest distance in the coordinate range as an end point to perform the processes of path planning, moving and the like, and performing related operations such as subsequent object searching and the like according to the coordinate range.
Fig. 9 is an architecture diagram of a robot spatial recognition system based on natural language interaction according to the present invention, as shown in fig. 9, the system includes: corpus creating unit 910, keyword determining unit 920, feature determining unit 930, and target object coordinate determining unit 940.
A corpus establishing unit 910, configured to establish a spatial information corpus based on natural language expression, including: a target attribute feature description corpus and a target location feature description corpus;
a keyword determining unit 920, configured to convert the target attribute feature description corpus and the target location feature description corpus into a keyword array according to a preset grammar rule; the keywords include: the target object name, the reference object name, the direction relation and the distance relation;
a feature determining unit 930, configured to determine, according to the object-related features included in the keyword array, a category and a spatial position calculation relationship, where the category includes a point object and a planar object, the spatial position calculation relationship includes at least one of the following relationships: the direction relation of the target object relative to the reference object, the distance relation of the target object relative to the reference object and the topological relation of the target object relative to at least two reference objects;
and an object coordinate determining unit 940, configured to determine a coordinate range of the object according to the spatial position calculation relationship and the category to which the object and the reference object belong, so as to perform object search in the following.
Alternatively, the keyword determination unit 920 determines the category to which the object belongs by: if the object of the category to be judged is an independent object, abstracting the object into a point object or a planar object without influencing the spatial position expression of the object or other objects except the object of the category to be judged, and regarding the object as a point object; if the area ratio of the object of the category to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted to be a point object and the spatial position expression of the object or other objects except the object of the category to be judged is influenced.
Optionally, according to the directional relationship of the object with respect to the reference object, the object coordinate determining unit solves the coordinate range of the object by: when the reference object is a point object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object at the origin of a coordinate system, and for any point-shaped target object in the space, obtaining a coordinate position set of the point-shaped target object relative to the point-shaped reference object in different directions according to a plurality of preset straight line constraints; when the reference object is a planar reference object, determining the planar reference object and a minimum circumscribed rectangle thereof by using a minimum boundary rectangle model, and taking straight lines where four rectangle sides of the minimum circumscribed rectangle are located as boundary lines in all directions; determining a coordinate position set of the point-shaped object relative to the planar reference object in different directions according to the boundary lines in all directions; if two reference objects exist, the coordinate position ranges of the target object are determined respectively only according to different reference object position descriptions, and then the intersection of the two ranges is obtained.
Optionally, according to a distance relationship between the target object and the reference object, the distance relationship includes: quantitative, qualitative, or temporal distances; the object coordinate determination unit solves the coordinate range of the object by the following steps: when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range area; when the distance relation is a qualitative distance, presetting different distance thresholds for distances of different granularity levels, and if the reference object is a point-shaped reference object, setting the distance of the point-shaped target object from the point-shaped reference object as a qualitative distance range area; when the distance relation is time distance, converting the time distance into quantitative distance, and then determining the coordinate range of the point-like reference object; when the distance relationship between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object need to be respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A robot space cognition method based on natural language interaction is characterized by comprising the following steps:
establishing a spatial information corpus based on natural language expression, comprising: a target attribute feature description corpus and a target location feature description corpus;
converting the target attribute feature description corpus and the target position feature description corpus into a keyword array according to a preset grammar rule; the keywords include: the target object name, the reference object name, the direction relation and the distance relation;
according to the related characteristics of the objects contained in the keyword array, judging the categories of the target object and the reference object and the spatial position calculation relationship, wherein the categories comprise a point object and a planar object, and the spatial position calculation relationship comprises at least one of the following relationships: the direction relation of the target object relative to the reference object, the distance relation of the target object relative to the reference object and the topological relation of the target object relative to at least two reference objects;
and determining the coordinate range of the target object according to the calculation relationship among the target object, the category of the reference object and the space position so as to search for the target object in the following process.
2. The robot space recognition method based on natural language interaction of claim 1, wherein the category to which the object belongs is determined by:
if the object of the category to be judged is an independent object, abstracting the object into a point object or a planar object without influencing the spatial position expression of the object or other objects except the object of the category to be judged, and regarding the object as a point object;
if the area ratio of the object of the category to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted to be a point object and the spatial position expression of the object or other objects except the object of the category to be judged is influenced.
3. The robot space recognition method based on natural language interaction of claim 2, wherein the coordinate range of the object is obtained by the following steps according to the directional relationship of the object with respect to the reference object:
when the reference object is a point object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object at the origin of a coordinate system, and for any point-shaped target object in the space, obtaining a coordinate position set of the point-shaped target object relative to the point-shaped reference object in different directions according to a plurality of preset straight line constraints;
when the reference object is a planar reference object, determining the planar reference object and a minimum circumscribed rectangle thereof by using a minimum boundary rectangle model, and taking straight lines where four rectangle sides of the minimum circumscribed rectangle are located as boundary lines in all directions; determining a coordinate position set of the point-shaped object relative to the planar reference object in different directions according to the boundary lines in all directions;
if two reference objects exist, the coordinate position ranges of the target object are determined respectively only according to different reference object position descriptions, and then the intersection of the two ranges is obtained.
4. The robot space recognition method based on natural language interaction of claim 3, wherein the distance relationship comprises, according to the distance relationship of the target object relative to the reference object: quantitative, qualitative, or temporal distances; solving the coordinate range of the object by the following steps:
when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range area;
when the distance relation is a qualitative distance, presetting different distance thresholds for distances of different granularity levels, and if the reference object is a point-shaped reference object, setting the distance of the point-shaped target object from the point-shaped reference object as a qualitative distance range area;
when the distance relation is time distance, converting the time distance into quantitative distance, and then determining the coordinate range of the point-like reference object;
when the distance relationship between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object need to be respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
5. The robot space recognition method based on natural language interaction of claim 4, wherein the coordinate range of the target object is obtained by the following steps according to the distance relationship and the direction relationship of the target object relative to the reference object:
and solving the coordinate range of the target object according to two constraint conditions of the distance relation and the direction relation of the target object relative to the reference object, and finally solving the intersection of the two coordinate ranges to determine the final coordinate range of the target object.
6. A robot space cognition method based on natural language interaction according to any of the claims 1 to 5, characterized in that, according to the topological relation of the object relative to at least two reference objects, the coordinate range of the object is solved by the following steps:
if the topological relation is that the target object is between the two reference objects:
when the two reference objects are both point reference objects, the target object is in the range of the line segment formed by connecting the two reference objects and the area with the distance line segment as the preset distance;
when the two reference objects are planar reference objects, the range of the target object is determined according to the rectangular edge of the minimum circumscribed rectangle of the two planar reference objects;
when the two reference objects are the point-like reference object and the planar reference object, the range of the target object is determined according to the coordinates of the point-like reference object and the rectangular side of the rectangular minimum circumscribed rectangle of the planar reference object.
7. A robotic space-aware system based on natural language interaction, comprising:
the corpus establishing unit is used for establishing a spatial information corpus based on natural language expression, and comprises the following steps: a target attribute feature description corpus and a target location feature description corpus;
a keyword determining unit, configured to convert the target attribute feature description corpus and the target location feature description corpus into a keyword array according to a preset grammar rule; the keywords include: the target object name, the reference object name, the direction relation and the distance relation;
a feature determination unit, configured to determine, according to the object-related features included in the keyword array, a category and a spatial position calculation relationship to which the target object and the reference object belong, where the category includes a point object and a planar object, and the spatial position calculation relationship includes at least one of the following relationships: the direction relation of the target object relative to the reference object, the distance relation of the target object relative to the reference object and the topological relation of the target object relative to at least two reference objects;
and the target object coordinate determining unit is used for determining the coordinate range of the target object according to the target object, the category of the reference object and the spatial position calculation relationship so as to search for the target object subsequently.
8. The robot-space recognition system based on natural language interaction of claim 7, wherein the keyword determination unit determines the category to which the object belongs by: if the object of the category to be judged is an independent object, abstracting the object into a point object or a planar object without influencing the spatial position expression of the object or other objects except the object of the category to be judged, and regarding the object as a point object; if the area ratio of the object of the category to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted to be a point object and the spatial position expression of the object or other objects except the object of the category to be judged is influenced.
9. The robot space recognition system based on natural language interaction of claim 8, wherein the object coordinate determination unit solves the coordinate range of the object by the following steps according to the directional relationship of the object with respect to the reference object: when the reference object is a point object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object at the origin of a coordinate system, and for any point-shaped target object in the space, obtaining a coordinate position set of the point-shaped target object relative to the point-shaped reference object in different directions according to a plurality of preset straight line constraints; when the reference object is a planar reference object, determining the planar reference object and a minimum circumscribed rectangle thereof by using a minimum boundary rectangle model, and taking straight lines where four rectangle sides of the minimum circumscribed rectangle are located as boundary lines in all directions; determining a coordinate position set of the point-shaped object relative to the planar reference object in different directions according to the boundary lines in all directions; if two reference objects exist, the coordinate position ranges of the target object are determined respectively only according to different reference object position descriptions, and then the intersection of the two ranges is obtained.
10. A robotic space-aware system based on natural language interaction as claimed in claim 9, wherein the distance relationship includes, in terms of the distance relationship of the object with respect to the reference: quantitative, qualitative, or temporal distances; the object coordinate determination unit solves the coordinate range of the object by the following steps: when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range area; when the distance relation is a qualitative distance, presetting different distance thresholds for distances of different granularity levels, and if the reference object is a point-shaped reference object, setting the distance of the point-shaped target object from the point-shaped reference object as a qualitative distance range area; when the distance relation is time distance, converting the time distance into quantitative distance, and then determining the coordinate range of the point-like reference object; when the distance relationship between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object need to be respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911207208.XA CN110990594B (en) | 2019-11-29 | 2019-11-29 | Robot space cognition method and system based on natural language interaction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911207208.XA CN110990594B (en) | 2019-11-29 | 2019-11-29 | Robot space cognition method and system based on natural language interaction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110990594A true CN110990594A (en) | 2020-04-10 |
| CN110990594B CN110990594B (en) | 2023-07-04 |
Family
ID=70088801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911207208.XA Active CN110990594B (en) | 2019-11-29 | 2019-11-29 | Robot space cognition method and system based on natural language interaction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110990594B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114064940A (en) * | 2021-11-12 | 2022-02-18 | 南京师范大学 | Fuzzy position description positioning method based on super-assignment semantics |
| CN114139069A (en) * | 2021-11-05 | 2022-03-04 | 深圳职业技术学院 | Indoor positioning method and system based on voice interaction and electronic equipment |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090192647A1 (en) * | 2008-01-29 | 2009-07-30 | Manabu Nishiyama | Object search apparatus and method |
| CN102915039A (en) * | 2012-11-09 | 2013-02-06 | 河海大学常州校区 | Multi-robot combined target searching method of animal-simulated space cognition |
| CN108377467A (en) * | 2016-11-21 | 2018-08-07 | 深圳光启合众科技有限公司 | Indoor positioning and interactive approach, the device and system of target object |
| CN108680163A (en) * | 2018-04-25 | 2018-10-19 | 武汉理工大学 | A kind of unmanned boat route search system and method based on topological map |
| WO2018191970A1 (en) * | 2017-04-21 | 2018-10-25 | 深圳前海达闼云端智能科技有限公司 | Robot control method, robot apparatus and robot device |
| CN109614550A (en) * | 2018-12-11 | 2019-04-12 | 平安科技（深圳）有限公司 | Public sentiment monitoring method, device, computer equipment and storage medium |
| CN109670262A (en) * | 2018-12-28 | 2019-04-23 | 江苏艾佳家居用品有限公司 | A kind of area of computer aided domestic layout optimization method and system |
| US20190122552A1 (en) * | 2017-06-19 | 2019-04-25 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for displaying a movement of a vehicle on a map |
| CN110019863A (en) * | 2017-12-26 | 2019-07-16 | 深圳市优必选科技有限公司 | Object searching method and device, terminal equipment and storage medium |
| CN110110823A (en) * | 2019-04-25 | 2019-08-09 | 浙江工业大学之江学院 | Object based on RFID and image recognition assists in identifying system and method |
-
2019
- 2019-11-29 CN CN201911207208.XA patent/CN110990594B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090192647A1 (en) * | 2008-01-29 | 2009-07-30 | Manabu Nishiyama | Object search apparatus and method |
| CN102915039A (en) * | 2012-11-09 | 2013-02-06 | 河海大学常州校区 | Multi-robot combined target searching method of animal-simulated space cognition |
| CN108377467A (en) * | 2016-11-21 | 2018-08-07 | 深圳光启合众科技有限公司 | Indoor positioning and interactive approach, the device and system of target object |
| WO2018191970A1 (en) * | 2017-04-21 | 2018-10-25 | 深圳前海达闼云端智能科技有限公司 | Robot control method, robot apparatus and robot device |
| US20190122552A1 (en) * | 2017-06-19 | 2019-04-25 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for displaying a movement of a vehicle on a map |
| CN110019863A (en) * | 2017-12-26 | 2019-07-16 | 深圳市优必选科技有限公司 | Object searching method and device, terminal equipment and storage medium |
| CN108680163A (en) * | 2018-04-25 | 2018-10-19 | 武汉理工大学 | A kind of unmanned boat route search system and method based on topological map |
| CN109614550A (en) * | 2018-12-11 | 2019-04-12 | 平安科技（深圳）有限公司 | Public sentiment monitoring method, device, computer equipment and storage medium |
| CN109670262A (en) * | 2018-12-28 | 2019-04-23 | 江苏艾佳家居用品有限公司 | A kind of area of computer aided domestic layout optimization method and system |
| CN110110823A (en) * | 2019-04-25 | 2019-08-09 | 浙江工业大学之江学院 | Object based on RFID and image recognition assists in identifying system and method |
Non-Patent Citations (2)
| Title |
|---|
| R MORATZ: "Intuitive Linguistic Joint Object Reference in Human-Robot Interaction" * |
| 李敏等: "基于四层树状语义模型的场景语义识别方法" * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114139069A (en) * | 2021-11-05 | 2022-03-04 | 深圳职业技术学院 | Indoor positioning method and system based on voice interaction and electronic equipment |
| CN114064940A (en) * | 2021-11-12 | 2022-02-18 | 南京师范大学 | Fuzzy position description positioning method based on super-assignment semantics |
| CN114064940B (en) * | 2021-11-12 | 2023-07-11 | 南京师范大学 | Fuzzy position description positioning method based on super assignment semantics |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110990594B (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111098301B (en) | Control method of task type robot based on scene knowledge graph | |
| Yao et al. | Fuzzy critical path method based on signed distance ranking of fuzzy numbers | |
| Casner | Task-analytic approach to the automated design of graphic presentations | |
| Yongda et al. | Research on multimodal human-robot interaction based on speech and gesture | |
| Khadraoui et al. | Visual servoing in robotics scheme using a camera/laser-stripe sensor | |
| KR20200054327A (en) | 3D space monitoring device, 3D space monitoring method, and 3D space monitoring program | |
| CN110990594B (en) | Robot space cognition method and system based on natural language interaction | |
| Liu et al. | Target localization in local dense mapping using RGBD SLAM and object detection | |
| CN114372173A (en) | Natural language target tracking method based on Transformer architecture | |
| CN112099627A (en) | City design virtual reality instant interaction platform based on artificial intelligence | |
| CN102306145A (en) | Robot navigation method based on natural language processing | |
| Toan et al. | Using modification of visibility-graph in solving the problem of finding shortest path for robot | |
| CN111159451B (en) | Power line point cloud dynamic monomer method based on spatial database | |
| CN116734856A (en) | Underwater robot path planning method considering ocean current influence | |
| Guo et al. | Study on landscape architecture model design based on big data intelligence | |
| Dichtl et al. | Polymap: A 2d polygon-based map format for multi-robot autonomous indoor localization and mapping | |
| Sheng et al. | CAD-guided sensor planning for dimensional inspection in automotive manufacturing | |
| Zhang et al. | Dynamic energy-efficient path planning of unmanned surface vehicle under time-varying current and wind | |
| CN117359643A (en) | Multi-modal element learning robot self-positioning system and method thereof | |
| Li et al. | Inferring user intent to interact with a public service robot using bimodal information analysis | |
| Skubic et al. | Generating linguistic spatial descriptions from sonar readings using the histogram of forces | |
| Lian et al. | Trajectory optimization of unmanned surface vehicle based on improved minimum snap | |
| Jia et al. | An improved semantic segmentation and fusion method for semantic SLAM | |
| Shen et al. | Robot Collaborative Interactive Artistic Design for Digital Art Using Embedded Systems | |
| Neto et al. | Data Structures in Robot Navigation Optimized by Adaptive Straightness |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |