CN110334114B - Method and device for inquiring data - Google Patents
Method and device for inquiring data Download PDFInfo
- Publication number
- CN110334114B CN110334114B CN201810145202.3A CN201810145202A CN110334114B CN 110334114 B CN110334114 B CN 110334114B CN 201810145202 A CN201810145202 A CN 201810145202A CN 110334114 B CN110334114 B CN 110334114B
- Authority
- CN
- China
- Prior art keywords
- jump
- data
- query
- linked list
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000004590 computer program Methods 0.000 claims description 9
- 230000009191 jumping Effects 0.000 claims description 8
- 239000000463 material Substances 0.000 description 48
- 230000008569 process Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24532—Query optimisation of parallel queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for inquiring data, and relates to the technical field of computers. One embodiment of the method comprises the following steps: receiving a query request, the query request comprising: inquiring the label; determining a jump linked list to be queried in a preset jump linked list set according to the query tag, wherein the jump linked list is used for storing data with the same query tag; and querying data in the jump linked list to be queried to obtain a query result. This embodiment is efficient and fast and has low performance overhead.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for querying data.
Background
In the process of processing and applying data, a many-to-many relationship between the data and the tag is often required to be established so as to query the data according to the tag. For example, when the e-commerce platform generates a feed stream of a commodity through soft text, evaluation, video, live broadcast and other materials related to the commodity, a many-to-many relationship between the material and a tag needs to be established, and when the feed stream of a certain tag is generated, the material is queried through the tag to generate the feed stream.
At present, a common method for inquiring data based on a many-to-many corresponding relation between the data and the tag is realized based on a relational database, and specifically has two realization schemes: one scheme is that a plurality of labels are combined into a field according to a stipulated mode, a certain column of data is stored, and when the data is inquired, an sql fuzzy inquiry statement is used for inquiring; another scheme is to build a table of association of data ids with tag ids and query the tag ids by means of table join.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art: when the fuzzy query is adopted, the query process cannot utilize indexes, the query efficiency is low, and the requirement of high availability under mass requests in the Internet age cannot be met. When adopting the table-linked query, the table-linked query can bring too high performance overhead under the condition of mass data of the internet content platform, and cause too much pressure on a database, so that the usability is limited.
Therefore, there is a need for a method and apparatus for querying data that is efficient, fast, and has low performance overhead.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a method and apparatus for querying data, which are efficient and fast, and have low performance overhead.
To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a method of querying data, including:
Receiving a query request, the query request comprising: inquiring the label;
Determining a jump linked list to be queried in a preset jump linked list set according to the query tag, wherein the jump linked list is used for storing data with the same query tag;
And querying data in the jump linked list to be queried to obtain a query result.
Further, the query request further includes: inquiring the characteristic range parameters of the object;
Determining the jump linked list to be queried in the preconfigured jump linked list set comprises the following steps:
Determining a jump link list subset of the query tag in the jump link list set according to the query tag, wherein the jump link list subset comprises: storing a jump chain table of data with the query tag;
And determining a jump linked list to be queried in the jump linked list subset according to the characteristic range parameters of the query object.
Further, determining the jump linked list to be queried in the jump linked list subset includes:
acquiring characteristic range parameters of each jump linked list in the jump linked list subset;
the following steps are executed for the characteristic range parameters of each jump linked list:
And judging whether the current characteristic range has an intersection with the characteristic range of the query object, if so, determining that the jump linked list corresponding to the current characteristic range parameter is the jump linked list to be queried.
Further, the method for querying data provided by the embodiment of the invention further comprises the following steps:
Receiving data to be stored;
determining a jump linked list subset for storing the data to be stored in the jump linked list set according to the label of the data to be stored;
Setting keywords for the data to be stored according to the characteristics of the data to be stored;
Determining a jump chain table for storing the data to be stored in the jump chain table subset according to the key words of the data to be stored;
and storing the data to be stored into the jump linked list according to the key words of the data to be stored.
Further, querying data in the jump linked list to be queried to obtain a query result includes:
obtaining a keyword range of the query object according to the characteristic range parameters of the query object;
Inquiring data in each jumping list to be inquired according to the keyword range of the inquired object so as to obtain the inquiring result of each jumping list to be inquired;
and merging all the query results of the jump linked list to be queried to obtain a final query result.
To achieve the above object, according to another aspect of an embodiment of the present invention, there is provided an apparatus for querying data, including:
The receiving module is used for receiving a query request, and the query request comprises: inquiring the label;
The determining module is used for determining a jump linked list to be queried in a preset jump linked list set according to the query tag, wherein the jump linked list is used for storing data with the same query tag;
And the query module is used for querying data in the jump linked list to be queried so as to obtain a query result.
Further, the query request further includes: inquiring the characteristic range parameters of the object;
The determining module is further configured to determine, according to the query tag, a subset of the hopping linked lists of the query tag in the hopping linked list set, where the subset of the hopping linked lists includes: and storing the jump linked list of the data with the query tag, and then determining the jump linked list to be queried in the jump linked list subset according to the characteristic range parameters of the query object.
Further, the determining module is further configured to obtain a characteristic range parameter of each of the subset of jump linked lists, and then perform the following steps on the characteristic range parameter of each of the jump linked lists:
And judging whether the current characteristic range has an intersection with the characteristic range of the query object, if so, determining that the jump linked list corresponding to the current characteristic range parameter is the jump linked list to be queried.
Further, the device for querying data provided by the embodiment of the invention further includes:
The storage module is used for receiving data to be stored, determining a jump link list subset for storing the data to be stored in the jump link list set according to the label of the data to be stored, setting keywords for the data to be stored according to the characteristics of the data to be stored, determining a jump link list for storing the data to be stored in the jump link list subset according to the keywords of the data to be stored, and storing the data to be stored in the jump link list according to the keywords of the data to be stored.
Further, the query module is further configured to obtain a keyword range of the query object according to the feature range parameter of the query object, then query data in each jump linked list to be queried according to the keyword range of the query object to obtain a query result of each jump linked list to be queried, and finally combine the query results of all jump linked lists to be queried to obtain a final query result.
To achieve the above object, according to another aspect of an embodiment of the present invention, there is provided an electronic device for querying data, including:
one or more processors;
Storage means for storing one or more programs,
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method for querying data provided by the embodiments of the present invention.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a computer-readable medium having stored thereon a computer program which, when executed by a processor, implements a method of querying data provided by the embodiments of the present invention.
According to the method and the device for inquiring the data, the data of different labels are respectively stored in different jump linked lists, when in inquiry, the jump linked list where the data are located is determined according to the labels, and then the data inquiry is carried out in the jump linked list, so that the data inquiry based on the many-to-many corresponding relation between the data and the labels is realized.
In the invention, a plurality of jump linked lists are provided for the same tag data in the jump linked list subset, so that the plurality of data of the same tag can be respectively stored in different jump linked lists, and all data are prevented from being stored in the same jump linked list, and when a hot key phenomenon occurs, an access overtime phenomenon caused by the bandwidth being make occurs. Meanwhile, when a plurality of jump linked lists to be queried exist, all the jump linked lists to be queried can be queried in parallel, so that the time complexity of query is reduced.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a main flow of a method for querying data according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an application flow of a method for querying data according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a main process for determining a list of hopping lists to be queried in a subset of hopping lists according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a main flow of querying data in a jump linked list to be queried according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of main modules of an apparatus for querying data according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a system for generating feed streams for query data provided by an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
Fig. 8 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
An embodiment of the present invention provides a method for querying data, as shown in fig. 1 and fig. 2, where the method includes: step S101, step S102, and step S103. In step S101, a query request is received, the query request including: the tag is queried. The user initiates the query request through the client, and the data corresponding to the label needs to be queried is indicated through the query label. In the invention, the data and the labels have a corresponding relation of many to many, namely, one piece of data can be marked with a plurality of labels, and one label can also be marked on a plurality of pieces of data. For example, in an application scenario of the e-commerce platform, the data may be material data of a feed stream, one piece of material data may have a plurality of dimension labels, such as a promotional activity label, a commodity type label, and the like, and the promotional activity label or the commodity type label may be marked on other material data with the same feature.
In step S101, according to the query tag, a jump link list to be queried is determined in a preconfigured jump link list set, wherein the jump link list is used for storing data with the same query tag. I.e. in the present invention a set of jump linked lists is preconfigured, comprising at least one jump linked list. The jump linked list skipList is a data structure for quickly searching elements in space change time, is a substitute data structure for a balance tree, and uses a random balance strategy to replace a strict forced tree balance strategy of the balance tree, so that the jump linked list has a simpler and effective insertion/deletion method, a faster search speed and a lower construction cost compared with a relational database.
In the present invention, data is stored in a jump chain table, the data stored in the jump chain table all have the same tag, and the jump chain table has a tag ID to externally represent the tag of the data stored therein. The jump linked list set can comprise a plurality of jump linked lists with different labels, and the jump linked list with the same label can also be a plurality of jump linked lists.
Step S103, inquiring data in the jump linked list to be inquired to obtain an inquiring result. And (3) performing data query in the jump linked list to be queried determined in the step S102 to obtain data of a corresponding tag, and then transmitting the result to the client for display to the user.
According to the method for inquiring the data, the data of different tags are respectively stored in different jump linked lists, when in inquiry, the jump linked list where the data is located is determined according to the tags, and then the data inquiry is carried out in the jump linked list, so that the data inquiry based on the many-to-many corresponding relation between the data and the tags is realized.
In the embodiment of the invention, the query request further comprises: the characteristic range parameter of the query object, namely the data which the user wants to query, uses the characteristic of the query object as the query condition, and in the invention, the characteristic of the data can be selected in combination with specific application scenes and requirements, for example, the time of data release or the heat of the data (the more browsing numbers, the higher heat) and the like. In step S102, the process of determining the to-be-queried jump linked list in the preconfigured jump linked list set specifically includes: first, determining a jump link list subset of the query tag in the jump link list set according to the query tag, wherein the jump link list subset comprises: the hopping lists storing the data with the query tag, i.e. all hopping lists in the subset of hopping lists of a certain query tag have the same tag ID, i.e. all data in the subset of hopping lists of a certain query tag are marked with the same tag. And then determining a jump linked list to be queried in the jump linked list subset according to the characteristic range parameters of the query object.
More specifically, in the embodiment of the present invention, as shown in fig. 3, the process of determining the to-be-queried jump linked list in the jump linked list subset is specifically: first, a characteristic range parameter for each of a subset of the hopping lists is obtained. In the invention, after the data is written into the jump linked list, the characteristic range of the data currently stored in the jump linked list is recorded to obtain the characteristic range parameter, and after the data is written or deleted, the characteristic range parameter is updated in real time. And when the data is queried, acquiring the current characteristic range parameters of the record, so as to know the characteristic ranges of all jump linked lists.
Then, traversing the characteristic range parameters of each jump linked list, and executing the following judging steps on the characteristic range parameters of each jump linked list: judging whether the current characteristic range and the characteristic range of the query object have intersection, if so, determining that the jump linked list corresponding to the current characteristic range parameter is the jump linked list to be queried, adding the jump linked list queue to be queried, and if not, discarding the jump linked list corresponding to the current characteristic range parameter. And finally judging whether the characteristic range parameters of all the jump linked lists are judged, if so, ending the process of determining the jump linked list to be inquired in the jump linked list subset, and if not, taking the characteristic range parameter of the next jump linked list in the jump linked list subset as the current characteristic range parameter, and repeatedly executing the steps until all the characteristic range parameters are judged.
In the invention, when the characteristic range of a certain jump linked list has intersection with the characteristic range of the query object, the jump linked list is indicated to store the data which the user wants to query, and the data is used as the jump linked list to be queried, and the data which the user wants to query can be obtained by querying the data.
In the invention, a plurality of jump linked lists are provided for the same tag data in the jump linked list subset, so that the plurality of data of the same tag can be respectively stored in different jump linked lists, and the occurrence of access timeout phenomenon caused by the bandwidth make when the hot key phenomenon occurs (for example, a certain key is accessed too much due to the online of a certain large theme activity of an e-commerce platform) is avoided. Meanwhile, when a plurality of jump linked lists to be queried exist, all the jump linked lists to be queried can be queried in parallel, so that the time complexity of query is reduced.
In the embodiment of the present invention, step S103, the process of querying data in the jump linked list to be queried to obtain the query result specifically includes: firstly, obtaining the keyword range of the query object according to the characteristic range parameter of the query object, wherein the keyword range of the query object is at least one keyword, and the keyword is the query condition of the query data in the jump chain table. And querying data in each jumping list to be queried according to the keyword range of the query object so as to obtain the query result of each jumping list to be queried. In the sub-set of jump linked lists, each keyword is unique, that is, the keywords of the data are not repeated among the jump linked lists with the same tag ID, so that the unique data corresponding to each keyword can be queried in the sub-set of jump linked lists according to the queried keyword range. When a plurality of jump linked lists to be queried exist, the step can query all the jump linked lists to be queried in a multithreaded parallel manner. And then merging the query results of all the jump linked lists to be queried to obtain the final complete query result.
In one embodiment of the present invention, as shown in fig. 4, the step S103 of performing multi-thread parallel query on all the jump linked lists to be queried and merging the query results may be implemented by a synchronization auxiliary class countdown latch in the java. First, the main thread initializes m countdown latch objects according to the number m of jumping-lists to be queried, and then the main thread Cheng Xinjian m sub-threads and passes the countdown latch objects to the sub-threads. Then, the main thread calls the await () method of the countdown latch object to suspend, waiting for the sub-thread to complete the task. The sub-threads concurrently perform tasks: each sub-thread accesses the corresponding jump linked list to perform data query, and after the task is executed, a countDown () method of the countdown latch object is called to reduce the value of m by 1, and when the value of m is reduced to 0, the result indicates that all the sub-threads execute the task. And judging whether all the sub threads execute the task according to the value of m, if not, continuing to execute the task by the sub threads, if so, waking up the suspended main thread, and carrying out the subsequent process of merging the query results by the main thread. In the embodiment of the invention, the query results of all the jump linked lists to be queried can be combined through a multi-path merging algorithm so as to obtain a final complete query result.
The method for inquiring the data provided by the embodiment of the invention further comprises the following process of storing the data into the jump linked list set: firstly, receiving data to be stored, and determining a jump linked list subset for storing the data to be stored in the jump linked list set according to the label of the data to be stored. Then, setting keywords for the data to be stored based on the characteristics of the data to be stored, and determining a jump chain table for storing the data to be stored in the jump chain table subset according to the keywords of the data to be stored. And finally, storing the data to be stored into the jump linked list.
In one embodiment of the present invention, for a plurality of pieces of data to be stored, the pieces of data can be distributed evenly to a plurality of pre-configured jump linked lists. For example, for a plurality of pieces of data to be stored, features (such as time features of the data) of each piece of data to be stored may be extracted, the features are processed according to a predefined keyword generation rule to generate keywords of the data, and then a hash value is calculated according to the keywords, so as to determine a jump chain table corresponding to the hash value, and the jump chain table is used for storing the piece of data to be stored.
The process of storing data into the jump linked list set can averagely distribute the data into a plurality of jump linked lists which are preconfigured, on one hand, the occurrence of access overtime phenomenon caused by a hot key phenomenon is solved, on the other hand, the time complexity of the jump linked list inquiry is reduced, and if the total data amount is N, the time complexity of the jump linked list inquiry is O (log (N)), and a plurality of (m) jump linked lists are adopted, and the data is averagely distributed into the m jump linked lists, so that the time complexity of inquiry is reduced to O (log (N/m)), and the response speed of inquiry is improved to a certain extent.
In the invention, the predefined keyword generation rules can be set in combination with specific application scenes and requirements. For example, when the extracted feature of the data to be stored is a time feature or a heat feature, the keyword generation rule may monotonically increase or decrease for the keyword according to the time or heat of the data to be stored.
In the invention, the number of the jump link list subsets and the number of the jump link list subsets in the jump link list set can be set according to specific application scenes and requirements. For example, in an electric commodity platform, the data queried by the invention is the material composing the feed stream, when a certain topic label is estimated to be developed to have the size of 1000 ten thousand materials after a few years, and meanwhile, one jump linked list is estimated according to a storage medium to store one hundred thousand materials, the number of the jump linked lists in a jump linked list subset of the topic label can be set to be 100.
The method for querying data provided by the embodiment of the invention is further described below with reference to a specific application scenario. In the application scene, the jump chain table set is used for storing ID information of materials of the feed stream, the ID information of the materials is provided with a pre-marked theme tag, in the example, the characteristics of the ID information of the materials are selected as the time of publishing the materials, the key words of the ID information of the materials are monotonically decreased according to the characteristics (the publishing time), namely, the earlier the publishing time is, the larger the key word value of the ID information is, the later the sorting in the jump chain table is, and in contrast, the least the key word value of the ID information is for the newly published materials, and the sorting in the jump chain table is the forefront.
In the process of storing ID information of the material, after the key words of the ID information of the material are determined, a plurality of jump linked lists with the ID of the subject tag in the jump linked list set are determined according to the subject tag of the material. Then, based on the key words, according to the hash algorithm, determining a jump chain table in which the ID information of the material is stored in the jump chain tables, storing the ID information of the material, and simultaneously updating the characteristic range parameters of the jump chain table (namely, the distribution time range of all the materials stored in the jump chain table) according to the characteristics of the jump chain table.
When a user selects a certain topic label at a mobile terminal and selects a time range (such as the last week) of a feed stream under the topic, the mobile terminal sends out a query request of the feed stream corresponding to the topic label, wherein the query request comprises the following steps: the subject tag and the feature range of the query object (the time range of the last week).
And determining a jump chain table with the ID of the topic tag according to the topic tag, then respectively judging whether the time range of the jump chain table and the time range of the query object are intersected, if so, indicating that the jump chain table stores the ID information of the material published in the time range of the last week, and taking the jump chain table as the jump chain table to be queried.
And inquiring the ID information of the materials from a plurality of jump linked lists to be inquired by utilizing multithreading in parallel, and combining the ID information of the materials by a main thread through a multi-way merging algorithm to obtain a final inquiring result. Because the ID information of the materials returned by each sub-thread is orderly, at the moment, the orderly arrangement of the ID information of all the materials forming the feed stream can be obtained by using a multi-path merging algorithm, the specific information of the materials can be stored in a redis database, and the specific information of the materials extracted from the redis database is returned to the client by utilizing the ID information of the materials in the final query result, so that the materials are formed into the feed stream and displayed to a user.
The embodiment of the invention also provides a device for querying data, as shown in fig. 5, the device 500 includes: a receiving module 501, a determining module 502 and a querying module 503.
The receiving module 501 is configured to receive a query request, where the query request includes: the tag is queried.
The determining module 502 is configured to determine, according to the query tag, a jump link list to be queried in a preconfigured jump link list set, where the jump link list is used to store data with the same query tag.
The query module 503 is configured to query data in a jump linked list to be queried to obtain a query result.
In the embodiment of the invention, the query request further comprises: and querying the characteristic range parameters of the object.
The determining module is further configured to determine, according to the query tag, a subset of the hopping list of the query tag in the hopping list set, where the subset of the hopping list includes: and storing the jump linked list of the data with the query tag, and then determining the jump linked list to be queried in the jump linked list subset according to the characteristic range parameters of the query object.
In the embodiment of the present invention, the determining module is further configured to obtain a characteristic range parameter of each of the subset of jump linked lists, and then perform the following steps on the characteristic range parameter of each of the jump linked lists:
And judging whether the current characteristic range has an intersection with the characteristic range of the query object, if so, determining that the jump linked list corresponding to the current characteristic range parameter is the jump linked list to be queried.
In the embodiment of the invention, the query module is further used for obtaining the keyword range of the query object according to the characteristic range parameter of the query object, then querying data in each jump linked list to be queried according to the keyword range of the query object to obtain the query result of each jump linked list to be queried, and finally merging the query results of all jump linked lists to be queried to obtain the final query result.
The device for inquiring the data provided by the embodiment of the invention further comprises: and a storage module. The storage module is used for receiving data to be stored, determining a jump link list subset for storing the data to be stored in the jump link list set according to the label of the data to be stored, setting keywords for the data to be stored according to the characteristics of the data to be stored, determining a jump link list for storing the data to be stored in the jump link list subset according to the keywords of the data to be stored, and storing the data to be stored in the jump link list according to the keywords of the data to be stored.
The embodiment of the invention also provides a system for generating the feed stream, which is applied to an application scene of inquiring materials according to the theme label to generate the feed stream, as shown in fig. 6, and comprises: the system comprises a material issuing unit, a material management unit, a stream computing unit, a storage unit and a multithreading unit.
The material publishing unit provides the creator of the material with the material for publishing, which is the source of all the materials of the system, and the published material is submitted to the material management unit for verification.
The material management unit is used by a service side providing feed stream and is used for managing materials, checking and uploading the released materials. The function of the storage module in the above embodiment of the invention is integrated, the published materials are received, after the materials are online, the materials are hashed into different jump linked lists, and meanwhile, the characteristic range parameters of each jump linked list are updated to the stream calculation module.
The functions of the receiving module and the determining module in the above embodiment of the present invention are integrated in the stream computing unit, and are used for maintaining the characteristic range parameters of each jump linked list, when the user sends a request through the client, the request of the user is received, the jump linked list which does not need to be accessed is filtered, and the jump linked list containing the required data is accurately positioned.
The storage unit is a unit for storing material-related data, which is implemented by a set of jump linked lists in the present invention. For example, as shown in the figure, the storage unit includes n jump linked lists skiplist to SKIPLIST N, each of which stores therein a plurality of pieces of data, each of which includes: key and value, the value of which is ID information of the material.
The multithreading unit integrates the function of the query module in the above embodiment of the present invention, which is used for taking data by multiple threads simultaneously, taking the data, and then assembling and returning the data in the main thread, aiming at the situation that the related data of the material in the storage unit is hashed to multiple jump linked lists.
According to the method and the device for inquiring the data, the data of different labels are respectively stored in different jump linked lists, when in inquiry, the jump linked list where the data are located is determined according to the labels, and then the data inquiry is carried out in the jump linked list, so that the data inquiry based on the many-to-many corresponding relation between the data and the labels is realized. In the invention, a plurality of jump linked lists are provided for the same tag data in the jump linked list subset, so that the plurality of data of the same tag can be respectively stored in different jump linked lists, and all data are prevented from being stored in the same jump linked list, and when a hot key phenomenon occurs, an access overtime phenomenon caused by the bandwidth being make occurs. Meanwhile, when a plurality of jump linked lists to be queried exist, all the jump linked lists to be queried can be queried in parallel, so that the time complexity of query is reduced.
Fig. 7 illustrates an exemplary system architecture 700 of a method of querying data or an apparatus of querying data to which embodiments of the present invention may be applied.
As shown in fig. 7, the system architecture 100 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 104 is used as a medium to provide communication links between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 705 via the network 704 using the terminal devices 701, 702, 703 to receive or send messages or the like. Various communication client applications, such as an application that sends a query data request to a server, may be installed on the terminal devices 701, 702, 703.
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server providing support for shopping-type websites browsed by the user using the terminal devices 701, 702, 703. The background management server can perform data query according to the received query data request and feed back the query result to the terminal equipment.
It should be noted that, the method for querying data provided by the embodiment of the present invention is generally executed by the server 705, and accordingly, the device for querying data is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, there is illustrated a schematic diagram of a computer system 800 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 8 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU) 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, mouse, etc.; an output portion 807 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 801.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a receiving module, a determining module, a querying module, and a storing module. The names of these modules do not in any way limit the module itself, for example, the determining module may also be described as "a module for obtaining a characteristic range parameter for each of the subset of jump links".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include:
Receiving a query request, the query request comprising: inquiring the label;
Determining a jump linked list to be queried in a preset jump linked list set according to the query tag, wherein the jump linked list is used for storing data with the same query tag;
And querying data in the jump linked list to be queried to obtain a query result.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of querying data, comprising:
receiving a query request, the query request comprising: inquiring the characteristic range parameters of the tag and the object;
Determining a jump linked list to be queried in a preset jump linked list set according to the query tag, wherein the jump linked list is used for storing data with the same query tag;
Querying data in a jump chain table to be queried to obtain a query result, including: obtaining a keyword range of the query object according to the characteristic range parameters of the query object; inquiring data in each jumping list to be inquired according to the keyword range of the inquired object so as to obtain the inquiring result of each jumping list to be inquired; and merging all the query results of the jump linked list to be queried to obtain a final query result.
2. The method of claim 1, wherein determining the hopping list to query from the set of preconfigured hopping lists comprises:
Determining a jump link list subset of the query tag in the jump link list set according to the query tag, wherein the jump link list subset comprises: storing a jump chain table of data with the query tag;
And determining a jump linked list to be queried in the jump linked list subset according to the characteristic range parameters of the query object.
3. The method of claim 2, wherein determining the hopping list to query in the subset of hopping lists comprises:
acquiring characteristic range parameters of each jump linked list in the jump linked list subset;
the following steps are executed for the characteristic range parameters of each jump linked list:
And judging whether the current characteristic range has an intersection with the characteristic range of the query object, if so, determining that the jump linked list corresponding to the current characteristic range parameter is the jump linked list to be queried.
4. A method according to claim 2 or 3, further comprising:
Receiving data to be stored;
determining a jump linked list subset for storing the data to be stored in the jump linked list set according to the label of the data to be stored;
Setting keywords for the data to be stored according to the characteristics of the data to be stored;
Determining a jump chain table for storing the data to be stored in the jump chain table subset according to the key words of the data to be stored;
and storing the data to be stored into the jump linked list according to the key words of the data to be stored.
5. An apparatus for querying data, comprising:
The receiving module is used for receiving a query request, and the query request comprises: inquiring the characteristic range parameters of the tag and the object;
The determining module is used for determining a jump linked list to be queried in a preset jump linked list set according to the query tag, wherein the jump linked list is used for storing data with the same query tag;
The query module is used for querying data in the jump linked list to be queried to obtain a query result, and comprises the following steps: obtaining a keyword range of the query object according to the characteristic range parameters of the query object; inquiring data in each jumping list to be inquired according to the keyword range of the inquired object so as to obtain the inquiring result of each jumping list to be inquired; and merging all the query results of the jump linked list to be queried to obtain a final query result.
6. The apparatus of claim 5, wherein the query request further comprises: inquiring the characteristic range parameters of the object;
The determining module is further configured to determine, according to the query tag, a subset of the hopping linked lists of the query tag in the hopping linked list set, where the subset of the hopping linked lists includes: and storing the jump linked list of the data with the query tag, and then determining the jump linked list to be queried in the jump linked list subset according to the characteristic range parameters of the query object.
7. The apparatus of claim 6, wherein the means for determining is further for obtaining a characteristic range parameter for each of the subset of the hopping lists, and then performing the following for the characteristic range parameter for each of the hopping lists:
And judging whether the current characteristic range has an intersection with the characteristic range of the query object, if so, determining that the jump linked list corresponding to the current characteristic range parameter is the jump linked list to be queried.
8. The apparatus according to claim 6 or 7, further comprising:
The storage module is used for receiving data to be stored, determining a jump link list subset for storing the data to be stored in the jump link list set according to the label of the data to be stored, setting keywords for the data to be stored according to the characteristics of the data to be stored, determining a jump link list for storing the data to be stored in the jump link list subset according to the keywords of the data to be stored, and storing the data to be stored in the jump link list according to the keywords of the data to be stored.
9. An electronic device for querying data, comprising:
one or more processors;
Storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-4.
10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810145202.3A CN110334114B (en) | 2018-02-12 | 2018-02-12 | Method and device for inquiring data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810145202.3A CN110334114B (en) | 2018-02-12 | 2018-02-12 | Method and device for inquiring data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110334114A CN110334114A (en) | 2019-10-15 |
| CN110334114B true CN110334114B (en) | 2024-05-17 |
Family
ID=68138801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810145202.3A Active CN110334114B (en) | 2018-02-12 | 2018-02-12 | Method and device for inquiring data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110334114B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113434572A (en) * | 2021-06-24 | 2021-09-24 | 中国工商银行股份有限公司 | Data query method, query system, device, storage medium and program product |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102479223A (en) * | 2010-11-25 | 2012-05-30 | ***通信集团浙江有限公司 | Data query method and system |
| CN103729471A (en) * | 2014-01-21 | 2014-04-16 | 华为软件技术有限公司 | Method and device for database query |
| CN103942289A (en) * | 2014-04-12 | 2014-07-23 | 广西师范大学 | Memory caching method oriented to range querying on Hadoop |
| CN104346362A (en) * | 2013-07-29 | 2015-02-11 | 腾讯科技（深圳）有限公司 | Method and device for finding target objects on basis of attribute values |
| CN106933914A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | The data processing method and device of many tables of data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9158801B2 (en) * | 2012-07-27 | 2015-10-13 | Facebook, Inc. | Indexing based on object type |
| US10997257B2 (en) * | 2015-02-06 | 2021-05-04 | Facebook, Inc. | Aggregating news events on online social networks |
-
2018
- 2018-02-12 CN CN201810145202.3A patent/CN110334114B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102479223A (en) * | 2010-11-25 | 2012-05-30 | ***通信集团浙江有限公司 | Data query method and system |
| CN104346362A (en) * | 2013-07-29 | 2015-02-11 | 腾讯科技（深圳）有限公司 | Method and device for finding target objects on basis of attribute values |
| CN103729471A (en) * | 2014-01-21 | 2014-04-16 | 华为软件技术有限公司 | Method and device for database query |
| CN103942289A (en) * | 2014-04-12 | 2014-07-23 | 广西师范大学 | Memory caching method oriented to range querying on Hadoop |
| CN106933914A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | The data processing method and device of many tables of data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110334114A (en) | 2019-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11422853B2 (en) | Dynamic tree determination for data processing | |
| US9996593B1 (en) | Parallel processing framework | |
| US8818940B2 (en) | Systems and methods for performing record actions in a multi-tenant database and application system | |
| CN109901987B (en) | Method and device for generating test data | |
| EP3093809B1 (en) | Systems and methods for state machine management | |
| CN110928853A (en) | Method and device for identifying log | |
| CN110650209A (en) | Method and device for realizing load balance | |
| CN111461583B (en) | Inventory checking method and device | |
| US9736082B2 (en) | Intelligent high-volume cloud application programming interface request caching | |
| CN110334114B (en) | Method and device for inquiring data | |
| CN112395337B (en) | Data export method and device | |
| US9805177B1 (en) | Processing large data sets from heterogeneous data sources using federated computing resources | |
| CN110347654B (en) | Method and device for online cluster characteristics | |
| CN112988806A (en) | Data processing method and device | |
| CN112784195A (en) | Page data publishing method and system | |
| US20220138799A1 (en) | Streamless content aware ad insertion | |
| CN112783904B (en) | Method and device for updating index data | |
| CN113760981B (en) | Data query method and device | |
| CN113741796B (en) | Data persistence method and device for terminal application | |
| CN114547455B (en) | Method and device for determining hot object, storage medium and electronic equipment | |
| CN112115165B (en) | Data processing method and device | |
| US20240202211A1 (en) | State rebalancing in structured streaming | |
| CN112667627B (en) | Data processing method and device | |
| CN111291038B (en) | Data query method and device | |
| CN113760925A (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |