CN112632304A - Data searching method and device based on index, server and storage medium - Google Patents
Data searching method and device based on index, server and storage medium Download PDFInfo
- Publication number
- CN112632304A CN112632304A CN202011645199.5A CN202011645199A CN112632304A CN 112632304 A CN112632304 A CN 112632304A CN 202011645199 A CN202011645199 A CN 202011645199A CN 112632304 A CN112632304 A CN 112632304A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- target
- clustering
- target file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000007613 environmental effect Effects 0.000 claims description 17
- 239000013598 vector Substances 0.000 claims description 14
- 239000000969 carrier Substances 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 7
- 238000003891 environmental analysis Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 abstract description 7
- 230000000875 corresponding effect Effects 0.000 description 130
- 230000001133 acceleration Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Probability & Statistics with Applications (AREA)
- Library & Information Science (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data searching method and device based on an index, a server and a storage medium, and belongs to the technical field of automatic driving. According to the method, the corresponding Btree index set is determined through the target movable carrier in the query information, then the target Btree index information corresponding to the query condition is determined from the Btree index set according to the dimension corresponding to the query condition, finally the corresponding data to be pushed is searched from the target Btree index information through the query condition, partial data meeting the user requirements are obtained through the target movable carrier and the query condition, the whole file corresponding to the movable carrier does not need to be downloaded, the partial data is positioned to be needed, and time cost and transmission cost can be effectively reduced.
Description
Technical Field
The invention relates to the technical field of automatic driving, in particular to a data searching method and device based on an index, a server and a storage medium.
Background
In the case of the automatic driving technology, since it needs to implement automatic navigation, it needs to continuously acquire video data around a movable carrier (for example, a vehicle, a robot with carrying capability, or an aircraft, etc.) through a camera, acquire point cloud data around the movable carrier through a radar, and acquire status data and positioning data of the movable carrier through a sensor, and the data acquired by the movable carrier may need to be used for data analysis, and therefore, it needs to be continuously uploaded to a server and stored in large files (the file size exceeds 1GB, and each large file corresponds to one movable carrier), and since each movable carrier generates data, the amount of the data is very large.
When a certain part of data of a certain movable carrier needs to be analyzed, the whole file corresponding to the movable carrier needs to be downloaded and the needed part of data needs to be relocated, and a large amount of time cost and transmission cost are wasted in the process. The existing solution usually processes the original data and stores it in another form in the database, and if the original file is not preserved, the data may be lost; if the original file is retained, the storage cost is doubled.
Disclosure of Invention
The invention mainly aims to provide a data searching method, a data searching device, a server and a storage medium based on indexes, and aims to solve the technical problem that time cost and transmission cost are too high when data are acquired in the prior art.
In order to achieve the above object, the present invention provides an index-based data searching method, which includes the following steps:
when receiving query information sent by a user terminal, extracting a target movable carrier and query conditions from the query information;
searching a Btree index set corresponding to the target movable carrier;
determining target Btree index information corresponding to the query condition from the Btree index set according to the dimensionality corresponding to the query condition;
and searching corresponding data to be pushed from the target Btree index information according to the query condition, and pushing the data to be pushed to a user terminal.
Optionally, before the step of finding the Btree index set corresponding to the target movable carrier, the index-based data finding method further includes:
acquiring image data, point cloud data, state data and positioning data corresponding to a movable carrier to be stored;
associating the image data, the point cloud data, the state data and the positioning data according to time, and taking the associated image data, point cloud data, state data and positioning data as target file information;
clustering data in the target file information based on a plurality of different dimensions, and generating Btree index information corresponding to the movable carrier to be stored in different dimensions according to a clustering result;
and adding the Btree index information respectively corresponding to the movable carriers to be stored in different dimensions into the Btree index set corresponding to the movable carriers to be stored.
Optionally, the step of clustering data in the target file information based on a plurality of different dimensions, and generating Btree index information corresponding to the to-be-stored movable carrier in different dimensions according to a clustering result specifically includes:
traversing a plurality of different dimensions, and taking the traversed dimension to be selected as the current dimension;
determining data to be clustered in the target file information based on the current dimension;
clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center;
and taking each cluster center as each primary key value in the Btree index information, and taking the cluster member corresponding to each cluster center as a leaf node corresponding to each primary key value in the Btree index information.
Optionally, the current dimension is an object query dimension, and the data to be clustered is image data and point cloud data;
the step of clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center specifically includes:
fusing image data and point cloud data of each target file information to obtain a fused image;
determining the outline of each object in the fused image according to the color difference between adjacent pixels;
carrying out object recognition on the fused image according to the outline of each object in the fused image so as to obtain the object position of each object in the fused image;
performing feature extraction based on the object position of each object in the fusion image to obtain feature information of each object;
and clustering the information of each target file based on the characteristic information to obtain each clustering center and clustering members of each clustering center.
Optionally, the current dimension is a state query dimension, and the data to be clustered is state data and positioning data;
the step of clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center specifically includes:
traversing the target file information, and taking the traversed target file information as the current target file information;
determining corresponding current theoretical state data based on current positioning data in the current target file information;
calculating a data difference value between the current state information and the current theoretical state data in the target file information;
generating a difference vector corresponding to the current target file information based on the data difference;
and after traversing the target file information, clustering the target file information based on the difference vector to obtain each cluster center and cluster members of each cluster center.
Optionally, the current dimension is a violation query dimension, and the data to be clustered is image data, state data and positioning data;
the step of clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center specifically includes:
traversing the target file information, and taking the traversed target file information as the current target file information;
mapping the current positioning data in the current target file information to a map;
determining a corresponding restriction status range based on current positioning data in the map;
comparing the current state data in the current target file information with the limited state range;
when the current state data is not in the limited state range, generating a violation type corresponding to the current target file information according to the current state data and the limited state range;
performing environmental analysis based on the image data to obtain environmental information around the movable carrier;
generating a violation reason according to the violation type and the environmental information;
and after traversing the target file information, clustering the target file information based on the violation reasons to obtain each clustering center and clustering members of each clustering center.
Optionally, the step of searching for corresponding data to be pushed from the target Btree index information through the query condition, and pushing the data to be pushed to the user terminal includes:
searching a target primary key value corresponding to the query condition from the target Btree index information;
determining a target leaf node corresponding to the target primary key value;
and taking target data corresponding to the clustering members in the target leaf nodes as data to be pushed, and pushing the data to be pushed to a user terminal.
In addition, to achieve the above object, the present invention further provides an index-based data search apparatus, including:
the information extraction module is used for extracting a target movable carrier and an inquiry condition from inquiry information when the inquiry information sent by a user terminal is received;
the set searching module is used for searching a Btree index set corresponding to the target movable carrier;
the information determining module is used for determining target Btree index information corresponding to the query condition from the Btree index set according to the dimensionality corresponding to the query condition;
and the data pushing module is used for searching corresponding data to be pushed from the target Btree index information through the query condition and pushing the data to be pushed to a user terminal.
In addition, to achieve the above object, the present invention provides a server, including: a memory, a processor, and an index-based data lookup program stored on the memory and executable on the processor, the index-based data lookup program configured to implement the steps of the index-based data lookup method as described above.
Further, to achieve the above object, the present invention also provides a computer readable storage medium having stored thereon an index-based data lookup program, which when executed by a processor, implements the steps of the index-based data lookup method as described above.
According to the method, the corresponding Btree index set is determined through the target movable carrier in the query information, then the target Btree index information corresponding to the query condition is determined from the Btree index set according to the dimension corresponding to the query condition, finally the corresponding data to be pushed is searched from the target Btree index information through the query condition, partial data meeting the user requirements are obtained through the target movable carrier and the query condition, the whole file corresponding to the movable carrier does not need to be downloaded, the partial data is positioned to be needed, and time cost and transmission cost can be effectively reduced.
Drawings
FIG. 1 is a flowchart illustrating a first embodiment of an index-based data lookup method according to the present invention;
FIG. 2 is a flowchart illustrating steps before step S20 in a second embodiment of the index-based data search method according to the present invention;
FIG. 3 is a flowchart illustrating step S03 of the index-based data searching method according to the third embodiment of the present invention;
FIG. 4 is a block diagram of an embodiment of an index-based data lookup apparatus according to the present invention;
fig. 5 is a schematic diagram of a server structure of a hardware operating environment according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a first embodiment of the index-based data searching method of the present invention.
In a first embodiment, the index-based data lookup method includes the steps of:
s10: when receiving query information sent by a user terminal, extracting a target movable carrier and query conditions from the query information.
It should be noted that the main execution body of the method of this embodiment is a server, and when a user terminal needs to query some data corresponding to a certain mobile carrier, a user may input a target mobile carrier and a query condition that need to be queried on the user terminal, at this time, the user terminal may generate a query message and send the query message to the server, so that, when receiving the query message sent by the user terminal, the server extracts the target mobile carrier and the query condition from the query message.
It can be understood that the user terminal is a terminal device of a user, which may be a smart phone, a notebook computer, a personal computer, or a tablet computer, and the present embodiment is not limited thereto.
In a specific implementation, the movable carrier has various expressions, such as a carrier with a moving capability, such as an automobile, a robot, an aircraft, etc., and the embodiment is not particularly limited.
S20: and searching a Btree index set corresponding to the target movable carrier.
It should be noted that, since each movable carrier can collect data separately, and the data are obviously different, the Btree index sets corresponding to each movable carrier can be generated separately based on the data of different movable carriers, so that after the target movable carrier is determined, the Btree index set corresponding to the target movable carrier can be searched.
S30: and determining target Btree index information corresponding to the query condition from the Btree index set according to the dimensionality corresponding to the query condition.
It is understood that for query conditions, corresponding query dimensions may differ for different query conditions, for example: when the query condition is data of a certain time period, the corresponding dimension of the query condition is a time dimension, but the query condition is assumed to be data of a certain location, and the corresponding query condition is a location dimension.
As can be seen from the above description, the indexes required for different query conditions are significantly different, and therefore, for data of the same movable carrier, Btree index information can be respectively established in advance according to the dimensions corresponding to the different query conditions, and the established Btree index information is added to the Btree index set, so that the target Btree index information corresponding to the query conditions can be determined from the Btree index set according to the dimensions corresponding to the query conditions.
In particular implementations, the Btree index information is an index data structure that features non-leaf nodes for describing the index, and leaf nodes pointing to specific data storage locations.
S40: and searching corresponding data to be pushed from the target Btree index information according to the query condition, and pushing the data to be pushed to a user terminal.
In a specific implementation, since the dimension of the target Btree index information is the same as the dimension of the query condition, in this embodiment, corresponding data to be pushed can be searched from the target Btree index information through the query condition, and the data to be pushed is pushed to the user terminal.
In the embodiment, a corresponding Btree index set is determined by a target movable carrier in query information, then the target Btree index information corresponding to the query condition is determined from the Btree index set according to the dimension corresponding to the query condition, finally, corresponding data to be pushed is searched from the target Btree index information according to the query condition, partial data meeting the user requirement is obtained through the target movable carrier and the query condition, a whole file corresponding to the movable carrier does not need to be downloaded, the partial data is relocated to be needed, and time cost and transmission cost can be effectively reduced.
As shown in fig. 2, a second embodiment of the index-based data lookup method according to the present invention is proposed based on the first embodiment, and in this embodiment, before step S20, the index-based data lookup method further includes:
s01: and acquiring image data, point cloud data, state data and positioning data corresponding to the movable carrier to be stored.
It should be noted that, the removable carrier to be stored is a removable carrier for which data storage is required.
It can be understood that, for the movable carrier, generally speaking, in order to implement automatic navigation, in the implementation process, the movable carrier needs to continuously acquire video data around the movable carrier through a camera, acquire point cloud data around the movable carrier through a radar, and acquire state data and positioning data of the movable carrier through a sensor, and these data need to be uploaded to a server by the movable carrier to be stored and stored by the server.
In a specific implementation, the image data, the point cloud data, the state data and the positioning data corresponding to the to-be-stored movable carrier are all within a preset time range, that is, the image data, the point cloud data, the state data and the positioning data respectively represent data of a plurality of different times.
S02: and associating the image data, the point cloud data, the state data and the positioning data according to time, and taking the associated image data, point cloud data, state data and positioning data as target file information.
It should be noted that, time corresponds to the image data, the point cloud data, the state data, and the positioning data, and these data need to be considered in combination when performing analysis, so in this embodiment, the image data, the point cloud data, the state data, and the positioning data may be associated according to time, and the associated image data, point cloud data, state data, and positioning data may be used as target file information.
Assuming that image data is { C1, C2, C3 … …, Cn }, point cloud data is { Q1, Q2, Q3 … …, Qn }, state data is { S1, S2, S3 … …, Sn }, and positioning data is { D1, D2, D3 … …, Dn }, where 1, 2, 3, … …, n represent different times, respectively, at this time, the image data, the point cloud data, the state data, and the positioning data may be associated according to time, for example: the target file information corresponding to time 1 is (C1, Q1, S1, D1), the target file information corresponding to time 2 is (C2, Q2, S2, D2), the target file information corresponding to time 3 is (C3, Q3, S3, D3), the target file information corresponding to time n is (… …), and the target file information corresponding to time n is (Cn, Q2, S2, D2).
S03: and clustering data in the target file information based on a plurality of different dimensions, and generating Btree index information corresponding to the movable carrier to be stored in different dimensions respectively according to a clustering result.
Because the data types required when generating the Btree index information with different dimensions are different, in this embodiment, the data in the target file information can be clustered based on a plurality of different dimensions, and the Btree index information corresponding to the to-be-stored movable carrier in different dimensions respectively is generated according to the clustering result.
S04: and adding the Btree index information respectively corresponding to the movable carriers to be stored in different dimensions into the Btree index set corresponding to the movable carriers to be stored.
In a specific implementation, since data of different removable carriers have differences, in this embodiment, Btree index information corresponding to the removable carriers to be stored in different dimensions respectively needs to be added to Btree index sets corresponding to the removable carriers to be stored.
According to the embodiment, image data, point cloud data, state data and positioning data corresponding to a movable carrier to be stored are correlated according to time, the correlated image data, point cloud data, state data and positioning data serve as target file information, then the data in the target file information are clustered based on a plurality of different dimensions, Btree index information corresponding to the movable carrier to be stored in different dimensions is generated according to clustering results, finally the Btree index information corresponding to the movable carrier to be stored in different dimensions is added to a Btree index set corresponding to the movable carrier to be stored, the data in the target file information are clustered based on different dimensions, and therefore the Btree index set corresponding to the movable carrier to be stored is generated, and subsequent data query is facilitated.
As shown in fig. 3, a third embodiment of the index-based data lookup method according to the present invention is proposed based on the first embodiment, and in this embodiment, step S03 specifically includes:
s031: and traversing a plurality of different dimensions, and taking the traversed dimension to be selected as the current dimension.
It should be noted that, for different dimensions, data clustering is respectively required, and therefore, in this embodiment, a plurality of different dimensions may be traversed, and a traversed dimension to be selected is taken as a current dimension.
S032: and determining the data to be clustered in the target file information based on the current dimension.
It can be understood that, for a dimension, different data to be clustered are generally used between different dimensions, and therefore, the data to be clustered in the target file information can be determined based on the current dimension.
S033: and clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center.
In a preferred embodiment of the present invention,
the first dimension is an object query dimension, and correspondingly, the data to be clustered are image data and point cloud data.
It can be understood that, in the case of automatic driving, since it is often necessary to analyze the driving state of the movable carrier when there are different types of objects in the vicinity of the vehicle, so as to adjust the automatic driving state thereof, in this embodiment, the step S033 may specifically include:
the method comprises the steps of firstly fusing image data and point cloud data of each piece of target file information to obtain a fused image, then determining the outline of each object in the fused image according to the color difference between adjacent pixels, then carrying out object identification on the fused image according to the outline of each object in the fused image to obtain the object position of each object in the fused image, then carrying out feature extraction based on the object position of each object in the fused image to obtain feature information of each object, and finally clustering each piece of target file information based on the feature information to obtain each clustering center and clustering members of each clustering center.
In the specific implementation, since the image data belongs to the data of two-dimensional layer, i.e. only the pixel coordinates and pixel color of each pixel can be reflected, but does not reflect the distance between the movable carrier and the object of each pixel, because, for the same object, the distance between the movable carrier and the object is different, the size of the objects in the image data may also differ, resulting in difficulty in accurately characterizing the objects even if the objects are determined in the image data, in order to compensate for this problem, in this embodiment, the image data and the point cloud data of each target file information are fused to obtain a fused image, and of course, the fusion of the image data and the point cloud data here refers to, and fusing the image data and the point cloud data in the same target file information, so that each target file information corresponds to a fused image.
It should be noted that, for the fused image, there may be a relatively large color difference between each object and other objects or background pixels, so in order to divide the objects in the fused image, in this embodiment, the contour of each object in the fused image may be determined according to the color difference between adjacent pixels, and then the object identification may be performed on the fused image according to the contour of each object in the fused image.
For different types of objects, the objects have different outlines, so a outline sample set can be established in advance, the outline sample set has outlines of different types of object samples at different angles, then the outlines of the objects in the fused image can be matched with the outline sample set, and the fused image is subjected to object identification through a matching result.
In a specific implementation, for the feature information, clustering may be performed on each target file information through the feature information, for example: for automobiles, vehicles with the same vehicle model can be classified into the same category, and for electric vehicles, electric vehicles with similar sizes can be classified into the same category.
For example: the fused image corresponding to the target file information 1 has the coincidence characteristic information A1Object X of1And conforming to the characteristic information B1Object Y of1And the sum conforms to the characteristic information C1Object Z of1The fused image corresponding to the target file information 2 has the coincidence characteristic information A2Object X of2And conforming to the characteristic information B2Object Y of2And the sum conforms to the characteristic information C2Object Z of2The fused image corresponding to the target file information 3 has the coincidence characteristic information A3Object X of3The fused image corresponding to the target file 4 has the coincidence characteristic information B4Object Y of4。
Hypothesis feature information A1Characteristic information A2And characteristic information A3Belonging to a relatively close feature, feature information B1Characteristic information B2And characteristic information B4Belonging to relatively close features, feature information C1And characteristic information C2Belonging to relatively similar characteristics, in which case, the characteristic information A can be used1As one of the clustering centers, and taking the target file information 1, the target file information 2 and the target file information 3 as clustering members; the characteristic information B is1As one of the clustering centers, and taking the target file information 1, the target file information 2 and the target file information 4 as clustering members; the characteristic information C is processed1As one of the cluster centers, and target file information 1 and target file information 2 as cluster members.
The second dimension is a state query dimension, and correspondingly, the data to be clustered are state data and positioning data.
It can be understood that, in the case of automatic driving, since it is often necessary to analyze data with a large difference between the state information of the movable carrier and the theoretical state data, in this embodiment, the step S033 may specifically include: traversing target file information, taking the traversed target file information as current target file information, determining corresponding current theoretical state data based on current positioning data in the current target file information, calculating a data difference value between the current state information and the current theoretical state data in the target file information, generating a difference value vector corresponding to the current target file information based on the data difference value, and clustering the target file information based on the difference value vector after traversing the target file information to obtain each clustering center and a clustering member of each clustering center.
In a specific implementation, the state data generally has data similar to a driving speed, a driving acceleration, a steering angle, and the like, and a corresponding theoretical state is generally set during the automatic driving process, that is, at a position corresponding to a certain positioning data, corresponding current theoretical state data is assigned to the certain positioning data during the automatic driving process, and it is assumed that the current theoretical state data assigned to the certain positioning data includes: the running speed is 50km/h, the running acceleration is 5km/h, and the steering angle is 5 degrees, but in the actual running process, the current state information comprises the following information: the running speed is 30km/h, the running acceleration is 10km/h, and the steering angle is 0 degree, at this time, a data difference between the current state information and the current theoretical state data in the target file information may be calculated, where the data difference is a running speed difference of-20 km/h, a running acceleration difference of 5km/h, and a steering angle difference of-5 degrees, and for convenience of subsequent processing, in this embodiment, normalization processing may be performed on the differences, for example: the running speed difference is normalized to-20, the running acceleration difference is normalized to 5, and the steering angle difference is normalized to-5, at which time a difference vector (-20, 5, -5) may be established based on the data difference.
For each piece of target file information, a difference vector is corresponding, and the difference vectors may correspond to a three-dimensional coordinate system, and when the target file information is clustered based on the difference vectors, the target file information may be clustered based on a distance between each difference vector and an origin of the three-dimensional coordinate system, at this time, each cluster center may be a distance between each difference vector and the origin of the three-dimensional coordinate system, and a cluster member corresponding to each cluster center is the corresponding target file information.
The third dimension is a violation query dimension, and correspondingly, the data to be clustered are image data, state data and positioning data.
It can be understood that, for the case of automatic driving, a situation of violating traffic rules sometimes occurs, and generally, the planning algorithm of automatic driving has some problems, so that it is often necessary to analyze and process the planning algorithm, and in order to perform data processing, in this embodiment, step S033 may specifically include: traversing the target file information, taking the traversed target file information as current target file information, mapping current positioning data in the current target file information to a map, determining a corresponding limited state range based on environmental information in the map and the current positioning data, comparing current state data in the current target file information with the limited state range, generating a violation type corresponding to the current target file information according to the current state data and the limited state range when the current state data is not in the limited state range, performing environmental analysis based on the image data to obtain environmental information around the movable carrier, generating a violation reason according to the violation type and the environmental information, and finally completing traversal of the target file information, and clustering the target file information based on the violation reasons to obtain each clustering center and clustering members of each clustering center.
It should be noted that, for different positions of the map, there will usually be corresponding limited state ranges, and assuming that the current positioning data is on the highway, at this time, the corresponding limited state ranges usually have a driving speed not lower than 60km/h and not higher than 120km/h, but assuming that the driving speed in the current state data of the movable carrier is 40km/h, and the current state data is not in the limited state ranges, at this time, the violation type corresponding to the current target file information can be generated according to the current state data and the limited state ranges as low-speed driving;
and then, assuming that the current positioning data is on a conventional road, at this time, the corresponding limited state range is that the running speed does not exceed 50km/h, but assuming that the running speed in the current running state of the movable carrier is 60km/h, at this time, the violation type corresponding to the current target file information can be generated according to the current state data and the limited state range and is overspeed running.
In a specific implementation, the environmental information may be information such as light intensity and shielding ratio;
taking the environmental information as the light intensity as an example, the light intensity can be determined based on the average brightness value through the average brightness value of each pixel in the image data;
taking the environmental information as the occlusion proportion as an example, the image data may be subjected to contour division to determine the position information of each object, determine the number of pixels corresponding to the position information of each object, calculate the proportion between the number of pixels and the total number of pixels of the image data, and take the maximum value of the proportion as the occlusion proportion.
It can be understood that, when the movable carrier is driven automatically, a violation condition usually does not occur, but sometimes an extreme environment occurs around the movable carrier, which affects a control instruction of automatic driving, and further causes a violation, so in this embodiment, environment analysis may be performed based on the image data to obtain environment information around the movable carrier.
Thus, after obtaining the environmental information around the movable carrier, the environmental information may be compared with standard environmental characteristics, and the target environmental information that does not match the standard environmental characteristics may be used as an influencing factor for generating a cause of violation, such as: when the movable carrier breaks rules and regulations at low speed, if the light intensity of the environment information is higher than the standard environment characteristics, the reason for breaking rules and regulations can be generated because the low-speed breaking rules and regulations are caused by overhigh light intensity.
Based on the mode, the target file information with the same violation causes can be clustered, namely, each clustering center can be the violation cause, and the clustering member corresponding to each clustering center is the target file information corresponding to each violation cause.
S034: and taking each cluster center as each primary key value in the Btree index information, and taking the cluster member corresponding to each cluster center as a leaf node corresponding to each primary key value in the Btree index information.
It should be noted that, because each cluster center is used as each primary key value in the Btree index information, and the cluster member corresponding to each cluster center is used as a leaf node corresponding to each primary key value in the Btree index information, in order to facilitate searching for data, step S40 may specifically include: searching a target primary key value corresponding to the query condition from the target Btree index information, then determining a target leaf node corresponding to the target primary key value, finally taking target data corresponding to a cluster member in the target leaf node as data to be pushed, and pushing the data to be pushed to a user terminal.
In the embodiment, the target file information is clustered through the data to be clustered to obtain each clustering center and clustering members of each clustering center, each clustering center is used as each primary key value in the Btree index information, and the clustering members corresponding to each clustering center are used as leaf nodes corresponding to each primary key value in the Btree index information, so that the Btree index information convenient for data query can be generated, and the efficiency and convenience of the data query are improved.
In addition, an embodiment of the present invention further provides an index-based data lookup apparatus, and with reference to fig. 4, the index-based data lookup apparatus includes:
an information extraction module 10, configured to, when query information sent by a user terminal is received, extract a target movable carrier and query conditions from the query information;
a set searching module 20, configured to search a Btree index set corresponding to the target movable carrier;
the information determining module 30 is configured to determine, according to the dimension corresponding to the query condition, target Btree index information corresponding to the query condition from the Btree index set;
and the data pushing module 40 is configured to search for corresponding data to be pushed from the target Btree index information according to the query condition, and push the data to be pushed to a user terminal.
According to the scheme, the corresponding Btree index set is determined by the target movable carrier in the query information, the target Btree index information corresponding to the query condition is determined from the Btree index set according to the dimension corresponding to the query condition, the corresponding data to be pushed is searched from the target Btree index information according to the query condition, partial data meeting the user requirement is obtained through the target movable carrier and the query condition, the whole file corresponding to the movable carrier does not need to be downloaded, the partial data is positioned to the required data, and time cost and transmission cost can be effectively reduced.
It should be noted that each module in the apparatus may be configured to implement each step in the method, and achieve the corresponding technical effect, which is not described herein again.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a server in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 5, the server may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the architecture shown in FIG. 5 does not constitute a limitation on servers, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 5, the memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and an index-based data lookup program.
In the server shown in fig. 5, the network interface 1004 is mainly used for data communication with an external network; the user interface 1003 is mainly used for receiving input instructions of a user; the removable carrier invokes an index-based data lookup procedure stored in memory 1005 by processor 1001 and performs the following operations:
when receiving query information sent by a user terminal, extracting a target movable carrier and query conditions from the query information;
searching a Btree index set corresponding to the target movable carrier;
determining target Btree index information corresponding to the query condition from the Btree index set according to the dimensionality corresponding to the query condition;
and searching corresponding data to be pushed from the target Btree index information according to the query condition, and pushing the data to be pushed to a user terminal.
Further, the processor 1001 may call an index-based data lookup procedure stored in the memory 1005, and also perform the following operations:
acquiring image data, point cloud data, state data and positioning data corresponding to a movable carrier to be stored;
associating the image data, the point cloud data, the state data and the positioning data according to time, and taking the associated image data, point cloud data, state data and positioning data as target file information;
clustering data in the target file information based on a plurality of different dimensions, and generating Btree index information corresponding to the movable carrier to be stored in different dimensions according to a clustering result;
and adding the Btree index information respectively corresponding to the movable carriers to be stored in different dimensions into the Btree index set corresponding to the movable carriers to be stored.
Further, the processor 1001 may call an index-based data lookup procedure stored in the memory 1005, and also perform the following operations:
traversing a plurality of different dimensions, and taking the traversed dimension to be selected as the current dimension;
determining data to be clustered in the target file information based on the current dimension;
clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center;
and taking each cluster center as each primary key value in the Btree index information, and taking the cluster member corresponding to each cluster center as a leaf node corresponding to each primary key value in the Btree index information.
Further, the current dimension is an object query dimension, and the data to be clustered is image data and point cloud data; the processor 1001 may call an index-based data lookup procedure stored in the memory 1005, and further perform the following operations:
fusing image data and point cloud data of each target file information to obtain a fused image;
determining the outline of each object in the fused image according to the color difference between adjacent pixels;
carrying out object recognition on the fused image according to the outline of each object in the fused image so as to obtain the object position of each object in the fused image;
performing feature extraction based on the object position of each object in the fusion image to obtain feature information of each object;
and clustering the information of each target file based on the characteristic information to obtain each clustering center and clustering members of each clustering center.
Further, the current dimension is a state query dimension, and the data to be clustered is state data and positioning data; the processor 1001 may call an index-based data lookup procedure stored in the memory 1005, and further perform the following operations:
traversing the target file information, and taking the traversed target file information as the current target file information;
determining corresponding current theoretical state data based on current positioning data in the current target file information;
calculating a data difference value between the current state information and the current theoretical state data in the target file information;
generating a difference vector corresponding to the current target file information based on the data difference;
and after traversing the target file information, clustering the target file information based on the difference vector to obtain each cluster center and cluster members of each cluster center.
Further, the current dimension is a violation query dimension, and the data to be clustered are image data, state data and positioning data; the processor 1001 may call an index-based data lookup procedure stored in the memory 1005, and further perform the following operations:
traversing the target file information, and taking the traversed target file information as the current target file information;
mapping the current positioning data in the current target file information to a map;
determining a corresponding restriction status range based on current positioning data in the map;
comparing the current state data in the current target file information with the limited state range;
when the current state data is not in the limited state range, generating a violation type corresponding to the current target file information according to the current state data and the limited state range;
performing environmental analysis based on the image data to obtain environmental information around the movable carrier;
generating a violation reason according to the violation type and the environmental information;
and after traversing the target file information, clustering the target file information based on the violation reasons to obtain each clustering center and clustering members of each clustering center.
Further, the processor 1001 may call an index-based data lookup procedure stored in the memory 1005, and also perform the following operations:
searching a target primary key value corresponding to the query condition from the target Btree index information;
determining a target leaf node corresponding to the target primary key value;
and taking target data corresponding to the clustering members in the target leaf nodes as data to be pushed, and pushing the data to be pushed to a user terminal.
According to the scheme, the corresponding Btree index set is determined by the target movable carrier in the query information, the target Btree index information corresponding to the query condition is determined from the Btree index set according to the dimension corresponding to the query condition, the corresponding data to be pushed is searched from the target Btree index information according to the query condition, partial data meeting the user requirement is obtained through the target movable carrier and the query condition, the whole file corresponding to the movable carrier does not need to be downloaded, the partial data is positioned to the required data, and time cost and transmission cost can be effectively reduced.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. An index-based data lookup method, comprising the steps of:
when receiving query information sent by a user terminal, extracting a target movable carrier and query conditions from the query information;
searching a Btree index set corresponding to the target movable carrier;
determining target Btree index information corresponding to the query condition from the Btree index set according to the dimensionality corresponding to the query condition;
and searching corresponding data to be pushed from the target Btree index information according to the query condition, and pushing the data to be pushed to a user terminal.
2. The index-based data lookup method of claim 1 wherein prior to the step of looking up the set of Btree indices corresponding to the target removable carrier, the index-based data lookup method further comprises:
acquiring image data, point cloud data, state data and positioning data corresponding to a movable carrier to be stored;
associating the image data, the point cloud data, the state data and the positioning data according to time, and taking the associated image data, point cloud data, state data and positioning data as target file information;
clustering data in the target file information based on a plurality of different dimensions, and generating Btree index information corresponding to the movable carrier to be stored in different dimensions according to a clustering result;
and adding the Btree index information respectively corresponding to the movable carriers to be stored in different dimensions into the Btree index set corresponding to the movable carriers to be stored.
3. The index-based data lookup method according to claim 2, wherein the step of clustering data in the target file information based on a plurality of different dimensions and generating Btree index information corresponding to the to-be-stored movable carrier in different dimensions respectively according to a clustering result specifically comprises:
traversing a plurality of different dimensions, and taking the traversed dimension to be selected as the current dimension;
determining data to be clustered in the target file information based on the current dimension;
clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center;
and taking each cluster center as each primary key value in the Btree index information, and taking the cluster member corresponding to each cluster center as a leaf node corresponding to each primary key value in the Btree index information.
4. The index-based data lookup method of claim 3 wherein the current dimension is an object query dimension, and the data to be clustered is image data and point cloud data;
the step of clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center specifically includes:
fusing image data and point cloud data of each target file information to obtain a fused image;
determining the outline of each object in the fused image according to the color difference between adjacent pixels;
carrying out object recognition on the fused image according to the outline of each object in the fused image so as to obtain the object position of each object in the fused image;
performing feature extraction based on the object position of each object in the fusion image to obtain feature information of each object;
and clustering the information of each target file based on the characteristic information to obtain each clustering center and clustering members of each clustering center.
5. The index-based data lookup method of claim 3 wherein the current dimension is a status query dimension, and the data to be clustered is status data and positioning data;
the step of clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center specifically includes:
traversing the target file information, and taking the traversed target file information as the current target file information;
determining corresponding current theoretical state data based on current positioning data in the current target file information;
calculating a data difference value between the current state information and the current theoretical state data in the target file information;
generating a difference vector corresponding to the current target file information based on the data difference;
and after traversing the target file information, clustering the target file information based on the difference vector to obtain each cluster center and cluster members of each cluster center.
6. The index-based data lookup method of claim 3 wherein the current dimension is a violation query dimension and the data to be clustered is image data, status data and positioning data;
the step of clustering the target file information based on the data to be clustered to obtain each clustering center and clustering members of each clustering center specifically includes:
traversing the target file information, and taking the traversed target file information as the current target file information;
mapping the current positioning data in the current target file information to a map;
determining a corresponding restriction status range based on current positioning data in the map;
comparing the current state data in the current target file information with the limited state range;
when the current state data is not in the limited state range, generating a violation type corresponding to the current target file information according to the current state data and the limited state range;
performing environmental analysis based on the image data to obtain environmental information around the movable carrier;
generating a violation reason according to the violation type and the environmental information;
and after traversing the target file information, clustering the target file information based on the violation reasons to obtain each clustering center and clustering members of each clustering center.
7. The index-based data searching method of any one of claims 3 to 6, wherein the step of searching the corresponding data to be pushed from the target Btree index information through the query condition and pushing the data to be pushed to a user terminal specifically comprises:
searching a target primary key value corresponding to the query condition from the target Btree index information;
determining a target leaf node corresponding to the target primary key value;
and taking target data corresponding to the clustering members in the target leaf nodes as data to be pushed, and pushing the data to be pushed to a user terminal.
8. An index-based data lookup apparatus, comprising:
the information extraction module is used for extracting a target movable carrier and an inquiry condition from inquiry information when the inquiry information sent by a user terminal is received;
the set searching module is used for searching a Btree index set corresponding to the target movable carrier;
the information determining module is used for determining target Btree index information corresponding to the query condition from the Btree index set according to the dimensionality corresponding to the query condition;
and the data pushing module is used for searching corresponding data to be pushed from the target Btree index information through the query condition and pushing the data to be pushed to a user terminal.
9. A server, characterized in that the server comprises: a memory, a processor, and an index-based data lookup program stored on the memory and executable on the processor, the index-based data lookup program configured to implement the steps of the index-based data lookup method of any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon an index-based data lookup program which, when executed by a processor, implements the steps of the index-based data lookup method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011645199.5A CN112632304B (en) | 2020-12-31 | 2020-12-31 | Index-based data searching method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011645199.5A CN112632304B (en) | 2020-12-31 | 2020-12-31 | Index-based data searching method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112632304A true CN112632304A (en) | 2021-04-09 |
| CN112632304B CN112632304B (en) | 2024-05-03 |
Family
ID=75291277
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011645199.5A Active CN112632304B (en) | 2020-12-31 | 2020-12-31 | Index-based data searching method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112632304B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706870A (en) * | 2021-08-30 | 2021-11-26 | 广州文远知行科技有限公司 | Method for collecting main vehicle lane change data in congested scene and related equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109408616A (en) * | 2018-10-10 | 2019-03-01 | 中南民族大学 | Content similarities short text querying method, equipment, system and storage medium |
| CN110209866A (en) * | 2019-05-30 | 2019-09-06 | 苏州浪潮智能科技有限公司 | A kind of image search method, device, equipment and computer readable storage medium |
| US20200004836A1 (en) * | 2018-06-03 | 2020-01-02 | Apple Inc. | Expanding indexed terms for searching files |
| CN110704453A (en) * | 2019-10-15 | 2020-01-17 | 腾讯音乐娱乐科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112100305A (en) * | 2020-09-15 | 2020-12-18 | 重庆中科云从科技有限公司 | Trajectory determination method, device, equipment and medium based on multi-source data |
-
2020
- 2020-12-31 CN CN202011645199.5A patent/CN112632304B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200004836A1 (en) * | 2018-06-03 | 2020-01-02 | Apple Inc. | Expanding indexed terms for searching files |
| CN109408616A (en) * | 2018-10-10 | 2019-03-01 | 中南民族大学 | Content similarities short text querying method, equipment, system and storage medium |
| CN110209866A (en) * | 2019-05-30 | 2019-09-06 | 苏州浪潮智能科技有限公司 | A kind of image search method, device, equipment and computer readable storage medium |
| CN110704453A (en) * | 2019-10-15 | 2020-01-17 | 腾讯音乐娱乐科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112100305A (en) * | 2020-09-15 | 2020-12-18 | 重庆中科云从科技有限公司 | Trajectory determination method, device, equipment and medium based on multi-source data |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706870A (en) * | 2021-08-30 | 2021-11-26 | 广州文远知行科技有限公司 | Method for collecting main vehicle lane change data in congested scene and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112632304B (en) | 2024-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4072173A1 (en) | Data transmission method and device | |
| US8929604B2 (en) | Vision system and method of analyzing an image | |
| WO2018229549A2 (en) | System and method for digital environment reconstruction | |
| WO2020258297A1 (en) | Image semantic segmentation method, movable platform, and storage medium | |
| CN113189989B (en) | Vehicle intention prediction method, device, equipment and storage medium | |
| CN112885134B (en) | Smart city traffic management method based on big data | |
| Houben et al. | Park marking-based vehicle self-localization with a fisheye topview system | |
| CN112632304B (en) | Index-based data searching method, device, server and storage medium | |
| CN110909656B (en) | Pedestrian detection method and system integrating radar and camera | |
| CN115147333A (en) | Target detection method and device | |
| CN114550117A (en) | Image detection method and device | |
| CN113297939A (en) | Obstacle detection method, system, terminal device and storage medium | |
| CN110555365A (en) | Distinguishing virtual objects from one another | |
| CN115861939A (en) | Method, device and equipment for inquiring vehicles in same line and storage medium | |
| CN115406452A (en) | Real-time positioning and mapping method, device and terminal equipment | |
| Ponn et al. | Performance Analysis of Camera-based Object Detection for Automated Vehicles. | |
| US11776200B2 (en) | Image relighting | |
| US20220228872A1 (en) | Apparatus and method for generating road map | |
| CN116168366B (en) | Point cloud data generation method, model training method, target detection method and device | |
| US20230244815A1 (en) | Anonymizing personally identifiable information in sensor data | |
| US11756261B2 (en) | Single-perspective image relighting | |
| CN114407916B (en) | Vehicle control and model training method and device, vehicle, equipment and storage medium | |
| CN114694375B (en) | Traffic monitoring system, traffic monitoring method, and storage medium | |
| US20240202533A1 (en) | Generating artificial video with changed domain | |
| Jukić et al. | Validation of Machine Learning-Based Lane Lines Detection Methods Using Different Datasets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |