CN111562898B - Multistage merging and sorting method based on FPGA - Google Patents
Multistage merging and sorting method based on FPGA Download PDFInfo
- Publication number
- CN111562898B CN111562898B CN202010391104.5A CN202010391104A CN111562898B CN 111562898 B CN111562898 B CN 111562898B CN 202010391104 A CN202010391104 A CN 202010391104A CN 111562898 B CN111562898 B CN 111562898B
- Authority
- CN
- China
- Prior art keywords
- data
- queue
- none
- last
- sorted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000009191 jumping Effects 0.000 claims description 48
- 238000012163 sequencing technique Methods 0.000 claims description 33
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000001174 ascending effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005587 bubbling Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/22—Arrangements for sorting or merging computer data on continuous record carriers, e.g. tape, drum, disc
- G06F7/24—Sorting, i.e. extracting data from one or more carriers, rearranging the data in numerical or other ordered sequence, and rerecording the sorted data on the original carrier or on a different carrier or set of carriers sorting methods in general
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a multistage merging and sorting method based on FPGA, which comprises the following steps: step 1, respectively detecting a plurality of 2-merging and sorting modules, wherein each 2-merging and sorting module comprises a queue A to be sorted and a queue B to be sorted; step 2, respectively carrying out merging and sorting on the plurality of 2 merging and sorting modules, and respectively outputting the arrangement results; and 3, taking the arrangement result as a queue to be ordered of the next stage, entering a 2-merging and ordering module of the next stage, and repeating the step 1 and the step 2 until a final arrangement result is output. According to the multistage merging and sorting method based on the FPGA, when the merging and sorting number is increased, the gate progression is constant, and merging can be carried out on sorting queues with any length.
Description
Technical Field
The invention relates to the field of computer communication, in particular to a multistage merging and sorting method based on FPGA.
Background
Ordering is a very important class of problems in the field of computer applications, where research and applications are very extensive. For example, ordering algorithms have found very wide application in data processing, databases, data compression, distributed computing, image processing, and computer graphics. Mainstream sorting algorithms are often developed based on a CPU (central processing unit), such as bubbling sorting, inserting sorting, binary tree sorting, quick sorting and the like, so that the efficiency of solving sorting problems is fully improved, the operation speed of the algorithm is improved, researchers design various sorting algorithms for different problems, and besides the parallelism characteristics of the sorting problems, a plurality of parallel sorting algorithms are designed by combining the structural characteristics of specific processors (CPUs, GPUs and FPGAs), wherein the parallel sorting algorithms based on the FPGAs are optimal in performance and power consumption ratio. The concept of Merge Sort (Merge Sort) is well suited for multi-threaded concurrent execution, and therefore Merge Sort is commonly used in the final stages of parallel Sort algorithms, and is therefore commonly employed in the digital circuit arts. The merging and sorting based on the FPGA is generally divided into two types according to different structures: a merge sort based on a sequential structure and a parallel merge sort based on a merge tree structure.
The merge ordering based on sequential structure proposed by bitton et al can order large data sets, but the ordering rate is usually 1 data per cycle, so low speed greatly limits the advantage of parallel processing of data by FPGA. Thus WeiSong, casper and Olukotun, etc. propose a parallel merge tree structure, based on which the parallel merge ordering is improved from 1 data per cycle to E data per sequence compared to conventional sequential merge ordering, and thus based on which the ordering is also referred to as E-data merge logic. Parallel merge ordering based on a merge tree structure can be divided into two models: the parallel merging tree structure proposed by WeiSong et al comprises two sets of input sequence FIFOs, two input managers and one combiner based on a biphone network; because of the complexity of dequeuing and reordering records from the input FIFO, it is difficult to run at a higher frequency when increasing the parallel ordering numbers. Casper and Olukotun et al, comprising two sets of input sequence FIFOs, a selector and a merging network based on parity merging network proposed by Batcher et al; this architecture increases the number of gate stages in the merging network as the number of parallel ranks increases, resulting in difficulty in operating at higher frequencies. The Casper and Olukotun model eliminates complex manager crossbars compared to the WeiSong model, facilitating high performance data ordering.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, the present invention aims to solve the technical problems of low processing speed and difficulty in running at high frequency with increasing of the number of sequences in the FPGA-based merging and sorting method in the prior art, and provide a multistage merging and sorting method based on FPGA, so that when the number of merging and sorting is increased, the gate progression is constant, and merging can be performed on sorting queues with any length.
In order to achieve the above purpose, the present invention provides a multistage merge ordering method implemented based on an FPGA, including the following steps:
and 3, taking the arrangement result as a queue to be ordered of the next stage, entering a 2-merging and ordering module of the next stage, and repeating the step 1 and the step 2 until a final arrangement result is output.
Further, the operation of simultaneous merging and sequencing of 2M input channels is realized by adopting a K-stage cascade mode, each stage comprises one or more 2 merging and sequencing modules, and each 2 merging and sequencing module of each stage adopts the same structure and realization method.
Further, in step 1, the first stage includes M2 merge sort modules, the 2 merge sort modules include a to-be-sorted queue a and a to-be-sorted queue B, the to-be-sorted queue a and the to-be-sorted queue B of the 1 st stage are set to be in a first data pre-reading mode, the to-be-sorted queue a and the to-be-sorted queue B respectively store data queues which are sorted according to a certain rule, and the to-be-sorted queue a and the to-be-sorted queue B can respectively output to-be-sorted data data_ A, data _b and a last flag bit last_ A, last _b of the present sorted data and a flag bit fifo_empty_ A, fifo _empty_b of the present to-be-sorted queue.
Further, the to-be-sorted queues a and B in all the M2 merge sort modules of the first stage must store at least one to-be-merged sort data; the to-be-ordered queue for each stage to 2-merge ordering must store any data with a last or none flag high.
Further, a to-be-sequenced queue A in the 2-merge sequencing module comprises to-be-sequenced data data_A, and to-be-sequenced data flag bits last_A and none_A; the to-be-sorted queue B comprises to-be-sorted data data_B, a to-be-sorted data flag bit last_B and a none_B; the sequencing result queue comprises sequenced data, a sequenced data flag bit last_flag and a none_flag;
the merging and sorting method of each 2-merging and sorting module comprises the following steps:
step 2-1), detecting a data_A flag bit none_A to be ordered, wherein the data_B flag bit none_B level to be ordered, if none_A is low level and none_B is low level, jumping to step 2-2); if none_A is high level and none_B is low level, jumping to step 2-7); if none_A is low level and none_B is high level, jumping to step 2-8); if none_A is high level, none_B is high level, jumping to step 2-9);
step 2-2), detecting last_A and last_B, if last_A is high level, last_B is low level, and jumping to step 2-4); if last_A is low level, last_B is high level, jump to step 2-5); if last_A is high level, last_B is high level, jump to step 2-6); if last_A is low level, last_B is low level, jump to step 2-3);
step 2-3), comparing the data to be sorted data_A with the data to be sorted data_B, and jumping to step 2-2) if the data_A is arranged in front and is written into the sorting result queue according to the set sorting rule A, and reading the next data data_A of the sorting result queue A to continuously compare with the data_B; otherwise, writing the data_B into the sequencing result queue, reading the next data data_B of the to-be-sequenced queue B, continuously comparing the next data data_B with the data_A, and jumping to the step 2-2);
step 2-4), comparing the data to be sorted data_A with the data to be sorted data_B, if the data_A is arranged in front, writing the data_A into a sorting result queue according to a set sorting rule A, then writing all data of the queue to be sorted B into the sorting result queue, setting last_flag as high level and none_flag as low level when last_B is high level, and jumping to step 2-1); otherwise, writing the data_B into the sequencing result queue, reading the next data data_B of the queue B to be sequenced, continuing to compare, and jumping to the step 2-2);
step 2-5), comparing the data to be sorted data_A with the data to be sorted data_B, if the data_B is arranged in front, writing the data_B into a sorting result queue according to a set sorting rule A, then writing all data of the queue to be sorted A into the sorting result queue, setting last_flag as high level and none_flag as low level when last_A is high level, and jumping to step 2-1); otherwise, writing the data_A into an ordering result queue, reading the next data data_A of the queue A to be ordered, continuing to compare, and jumping to the step 2-2);
step 2-6), comparing the data to be sorted data_A with the data to be sorted data_B, and jumping to step 2-1) according to a set sorting rule A if the data_A is arranged in front and is written into a sorting result queue, then writing the data_B into the sorting result queue and setting last_flag to be high level and none_flag to be low level; otherwise, writing the data_B into the sequencing result queue, then writing the data_A into the sequencing result queue, setting last_flag to be high level, setting none_flag to be low level, and jumping to the step 2-1);
step 2-7), reading an empty queue A to be ordered, writing all data of the queue B to be ordered into an ordering result queue, setting last_flag to be high level when last_B is high level, setting none_flag to be low level, and jumping to step 2-1);
step 2-8), reading an empty queue B to be ordered, writing all data of the queue A to be ordered into an ordering result queue, setting last_flag to be high level when last_A is high level, setting none_flag to be low level, and jumping to step 2-1);
step 2-9), writing the data_A into the sequencing result queue, setting the last_flag to be high level, setting the none_flag to be high level, reading the empty queue A to be sequenced and the queue B to be sequenced, and jumping to the step 2-1).
Further, none_A is high level, none_B is low level, only data_B is written into the sequencing result queue; the none_B is high level, the none_A is low level, and only the data_A is written into the sequencing result queue; both none_A and none_B are high, only data_A is written into the ordering result queue.
Further, the data length of the to-be-sorted queue a and the to-be-sorted queue B in the 2-merge sort module may be arbitrary.
Technical effects
The invention provides a multi-stage merging and sorting method based on FPGA, which supports sorting queues with any length and sorting data with large data volume, realizes merging of two sorting queues with any length into one ordered queue through reading control, and meets high-efficiency and rapid data sorting of massive data while meeting time sequence requirements through multi-stage data merging and sorting. The defect that the two groups of input queues have the same length in the traditional FPGA-based merging and sorting implementation method is overcome, and the comparison of any lengths of the two groups of comparison queues is realized.
The conception, specific structure, and technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, features, and effects of the present invention.
Drawings
FIG. 1 is a schematic diagram of a system framework of a multi-stage merge ordering method implemented based on an FPGA in accordance with a preferred embodiment of the present invention;
FIG. 2 is a schematic diagram of a process flow of a 2-merge ordering module of a multi-stage merge ordering method implemented based on an FPGA according to a preferred embodiment of the invention;
FIG. 3 is a waveform diagram of a first embodiment of the present invention;
FIG. 4 is a waveform diagram of a second embodiment of the present invention;
fig. 5 is a waveform diagram of a second embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects to be solved more clear, the invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular internal procedures, techniques, etc. in order to provide a thorough understanding of embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
As shown in fig. 1, the invention provides a multistage merging and sorting method based on FPGA, which comprises the following steps:
and 3, taking the arrangement result as a queue to be ordered of the next stage, entering a 2-merging and ordering module of the next stage, and repeating the step 1 and the step 2 until a final arrangement result is output.
As shown in FIG. 1, the multi-stage merging and sorting method based on FPGA disclosed by the invention adopts a K-stage cascade mode to realize the operation of merging and sorting 2M input channels simultaneously, each stage comprises one or more 2 merging and sorting modules, and each 2 merging and sorting module of each stage adopts the same structure and implementation method. The first stage includes M2 merge sort modules, where the 2 merge sort modules include a to-be-sorted queue a and a to-be-sorted queue B, the to-be-sorted queue a and the to-be-sorted queue B of the 1 st stage are set to be in a first data pre-reading mode, and the to-be-sorted queue a and the to-be-sorted queue B respectively store data queues that have been sorted according to a certain rule, and can respectively output to-be-sorted data data_ A, data _b, a last flag bit last_ A, last _b of the to-be-sorted data, a flag bit none_ A, none _b of the to-be-sorted data, and a flag bit fifo_empty_ A, fifo _empty_b of the to-be-sorted queue.
All the M2 merging and sorting modules of the first stage must store at least one data to be merged and sorted in a queue A to be sorted and a queue B to be sorted; the queue to be ordered, which performs 2-merge ordering at each stage, must store any data with a last or none flag of high level, when the queue data to be ordered are all invalid data, the none flag is of high level, otherwise the none flag is of low level, when the queue data to be ordered are last data, the last flag is of high level, otherwise the last flag is of low level.
The to-be-sequenced queue A in the 2-merging and sequencing module comprises to-be-sequenced data data_A, and to-be-sequenced data flag bits last_A and none_A; the to-be-sorted queue B comprises to-be-sorted data data_B, a to-be-sorted data flag bit last_B and a none_B; the sequencing result queue comprises sequenced data, a sequenced data flag bit last_flag and a none_flag; when the queue data to be ordered are all invalid data, the none mark is high level, otherwise, the none mark is low level; when the data of the queue to be ordered is the last data, the last mark is high level, otherwise, the last mark is low level.
The merging and sorting method of each 2-merging and sorting module comprises the following steps:
step 2-1), detecting a data_A flag bit none_A to be ordered, wherein the data_B flag bit none_B level to be ordered, if none_A is low level and none_B is low level, jumping to step 2-2); if none_A is high level and none_B is low level, jumping to step 2-7); if none_A is low level and none_B is high level, jumping to step 2-8); if none_A is high level, none_B is high level, jumping to step 2-9);
step 2-2), detecting last_A and last_B, if last_A is high level, last_B is low level, and jumping to step 2-4); if last_A is low level, last_B is high level, jump to step 2-5); if last_A is high level, last_B is high level, jump to step 2-6); if last_A is low level, last_B is low level, jump to step 2-3);
step 2-3), comparing the data to be sorted data_A with the data to be sorted data_B, and jumping to step 2-2) if the data_A is arranged in front and is written into the sorting result queue according to the set sorting rule A, and reading the next data data_A of the sorting result queue A to continuously compare with the data_B; otherwise, writing the data_B into the sequencing result queue, reading the next data data_B of the to-be-sequenced queue B, continuously comparing the next data data_B with the data_A, and jumping to the step 2-2);
step 2-4), comparing the data to be sorted data_A with the data to be sorted data_B, if the data_A is arranged in front, writing the data_A into a sorting result queue according to a set sorting rule A, then writing all data of the queue to be sorted B into the sorting result queue, setting last_flag as high level and none_flag as low level when last_B is high level, and jumping to step 2-1); otherwise, writing the data_B into the sequencing result queue, reading the next data data_B of the queue B to be sequenced, continuing to compare, and jumping to the step 2-2);
step 2-5), comparing the data to be sorted data_A with the data to be sorted data_B, if the data_B is arranged in front, writing the data_B into a sorting result queue according to a set sorting rule A, then writing all data of the queue to be sorted A into the sorting result queue, setting last_flag as high level and none_flag as low level when last_A is high level, and jumping to step 2-1); otherwise, writing the data_A into an ordering result queue, reading the next data data_A of the queue A to be ordered, continuing to compare, and jumping to the step 2-2);
step 2-6), comparing the data to be sorted data_A with the data to be sorted data_B, and jumping to step 2-1) according to a set sorting rule A if the data_A is arranged in front and is written into a sorting result queue, then writing the data_B into the sorting result queue and setting last_flag to be high level and none_flag to be low level; otherwise, writing the data_B into the sequencing result queue, then writing the data_A into the sequencing result queue, setting last_flag to be high level, setting none_flag to be low level, and jumping to the step 2-1);
step 2-7), reading an empty queue A to be ordered, writing all data of the queue B to be ordered into an ordering result queue, setting last_flag to be high level when last_B is high level, setting none_flag to be low level, and jumping to step 2-1);
step 2-8), reading an empty queue B to be ordered, writing all data of the queue A to be ordered into an ordering result queue, setting last_flag to be high level when last_A is high level, setting none_flag to be low level, and jumping to step 2-1);
step 2-9), writing the data_A into the sequencing result queue, setting the last_flag to be high level, setting the none_flag to be high level, reading the empty queue A to be sequenced and the queue B to be sequenced, and jumping to the step 2-1).
The none_A is high level, the none_B is low level, and only the data_B is written into the sequencing result queue; the none_B is high level, the none_A is low level, and only the data_A is written into the sequencing result queue; both none_A and none_B are high, only data_A is written into the ordering result queue.
The data length of the to-be-sorted queue a and the to-be-sorted queue B in the 2-merge sort module may be arbitrary.
The implementation of the present invention will be explained in 3 specific embodiments, and embodiments one to three respectively explain the multi-stage merge ordering process of the to-be-ordered queues a and B with different data lengths.
In 2-merge ordering of data of a database, in a typical SQL query language, there may be several language elements, such as SELECT FROM, WHERE, GROUP BY and ORDER BY. The SELECT statement retrieves the required data column from a given table. The WHERE statement is used to limit the data in the table and includes operations such as logical comparisons and arithmetic operations that filter the data that is relevant to the user. The GROUP BY statement GROUPs data into GROUPs based on a given field, and the ORDER BY statement ORDERs the data in ascending or descending ORDER according to a given key.
Example 1
As shown in fig. 3, this embodiment takes the secondary merge ordering of 4 sets of data in the database as an example, and the queues to be ordered 1_a, 1_B, 2_A, 2_B are subjected to ascending merge ordering.
1. At the first stage of merging: the queue 1_A to be ordered comprises data data_A {1,2,3}, and an ordered data flag bit last_A {1,0} and none_A {0,0}; the to-be-sorted queue 1_B contains to-be-sorted data data_b {4,5}, sorting data flag bits last_b {1,0} and none_b {0,0};
(1) Detecting that none_A and none_B corresponding to data_A:3 and data_B:5 are both in low level, namely that the queue 1_A to be ordered and the queue 1_B to be ordered are both effective data;
(2) Comparing the data to be ordered data_A with the queue to be ordered B, wherein the data_A is 3< data_B:5, the data_B:5 is written into the ordering result, then the data_B is read out and the data_B is 4, and the data_B is 4> data_A:3, and then the data_B is written into the ordering result;
(3) The flag bit last_B corresponding to the data_B:4 is detected to be at a high level, namely the data_B:4 is the last data of the queue 1_B to be ordered and is written into the result queue; the flag bit last_A corresponding to the data_A:3 is in a low level, namely the data of the queue 1_A to be ordered is not read;
(4) Reading all data data_A {1,2,3} of the queue 1_A to be ordered according to the method, writing the data data_A {1,2,3} into the ordering result queue, when the data_A is read out, when the data_A is detected to be high level, setting the last_A 1 into the ordering result queue, and setting the none_flag into low level;
(5) The merged ordered data is: {1,2,3,4,5}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0,0,0,0}.
And (3) the same principle: for the first level merge sort: the to-be-sorted queue 2_A contains to-be-sorted data data_A {8,9}, sorting data flag bit last_A {1,0} and none_A {0,0}; the to-be-sorted queue 2_B contains to-be-sorted data data_b {7}, sorting data flag bits last_b {1} and none_b {1};
(1) Data_a is detected: the 9 corresponding zone bit none_A is low level, namely the queue 2_A to be ordered is valid data, and the data_B is 7 corresponding zone bit none_B is high level, namely the queue 2_B to be ordered is invalid data;
(2) Reading the empty queue 2_B to be ordered, reading all data data_A {8,9} of the queue 2_A to be ordered according to the read sequence, writing the data data_A {8,9} into the ordering result queue, when the data_A:8 is read, detecting that the last_A is high level, and when the data_A:8 is written into the ordering result queue, setting the last_flag to be high level, and setting the none_flag to be low level;
(3) The merged ordered data is: {8,9}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0}.
2. The result sequencing queue sequenced data generated by completing the first-stage merging sequencing of the to-be-sequenced queue 1_A and the to-be-sequenced queue 1_B is as follows: {1,2,3,4,5}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0} is input to the second-stage merge sort waiting queue 1_A; the result ordering queue ordered data generated by completing the first-stage merge ordering of the to-be-ordered queue 2_A and the to-be-ordered queue 2_B is: {8,9}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0} is input to the second-stage merge sort pending queue 1_B;
for the second level merge sort: the queue 1_A to be ordered comprises data_A {1,2,3,4,5} to be ordered, and an ordering data flag bit last_A {1,0} and none_A {0,0}; the to-be-sorted queue 1_B contains to-be-sorted data data_b {8,9}, sorting data flag bits last_b {1,0} and none_b {0,0};
(1) Data_a is detected: the 5 corresponding zone bit none_A is low level, namely the queue 2_A to be ordered is valid data, and the 9 corresponding zone bit none_B is low level, namely the queue 2_B to be ordered is valid data;
(2) Comparing the data to be sorted data_A with a queue to be sorted B, wherein the data_A is 5< data_B:9, the data_B:9 is written into a sorting result, then the data_B is read out by 8, and the data_B is 8> data_A:5, and then the data_B is written into the sorting result by 8;
(3) The flag bit last_B corresponding to the data_B8 is detected to be at a high level, namely the data_B8 is the last data of the queue 1_B to be ordered and is written into the result queue; the flag bit last_A corresponding to the data_A:5 is in a low level, namely the data of the queue 1_A to be ordered is not read;
(4) Reading all data data_A {1,2,3,4,5} of the queue 1_A to be ordered according to the method, writing the data data_A {1,2,3,4,5} into the ordering result queue, when the data_A is read out, detecting that the last_A is high level, and when the data_A is written into the ordering result queue, setting the last_flag to be high level, and setting the none_flag to be low level;
(5) The merged ordered data is: {1,2,3,4,5,8,9}, corresponding last_flag is: {1,0,0,0,0,0,0}, corresponding none_flag is: {0,0,0,0,0,0,0}.
Second embodiment
As shown in fig. 4, this embodiment takes the secondary merge ordering of 4 sets of data in the database as an example, and the queues to be ordered 1_a, 1_B, 2_A, 2_B are subjected to ascending merge ordering.
At the first stage of merging: the queue 1_A to be ordered comprises data data_A {1,2,3}, and an ordered data flag bit last_A {1,0} and none_A {0,0}; the to-be-sorted queue 1_B contains to-be-sorted data data_b {4,5}, sorting data flag bits last_b {1,0} and none_b {0,0};
(1) Detecting that none_A and none_B corresponding to data_A:3 and data_B:5 are both in low level, namely that the queue 1_A to be ordered and the queue 1_B to be ordered are both effective data;
(2) Comparing the data to be ordered data_A with the queue to be ordered B, wherein the data_A is 3< data_B:5, the data_B:5 is written into the ordering result, then the data_B is read out and the data_B is 4, and the data_B is 4> data_A:3, and then the data_B is written into the ordering result;
(3) The flag bit last_B corresponding to the data_B:4 is detected to be at a high level, namely the data_B:4 is the last data of the queue 1_B to be ordered and is written into the result queue; the flag bit last_A corresponding to the data_A:3 is in a low level, namely the data of the queue 1_A to be ordered is not read;
(4) Reading all data data_A {1,2,3} of the queue 1_A to be ordered according to the method, writing the data data_A {1,2,3} into the ordering result queue, when the data_A is read out, when the data_A is detected to be high level, setting the last_A 1 into the ordering result queue, and setting the none_flag into low level;
(5) The merged ordered data is: {1,2,3,4,5}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0,0,0,0}.
And (3) the same principle: for the first level merge sort: the to-be-sorted queue 2_A contains to-be-sorted data data_A {9}, sorting data flag bits last_A {1} and none_A {1}; the to-be-sorted queue 2_B contains to-be-sorted data data_b {7}, sorting data flag bits last_b {1} and none_b {1};
(1) Data_a is detected: the 9 corresponding zone bit none_A is high level, namely the queue 2_A to be ordered is valid data, and the data_B is 7 corresponding zone bit none_B is high level, namely the queue 2_B to be ordered is invalid data;
(2) Reading the empty queue 2_B to be ordered and the queue 2_A to be ordered; writing data_A {9} into the sequencing result queue, and setting last_flag to be high level and none_flag to be high level;
(3) The merged ordered data is: {9}, corresponding last_flag is: {1}, corresponding none_flag is: {1}.
For the second level merge sort: the queue 1_A to be ordered comprises data_A {1,2,3,4,5} to be ordered, and an ordering data flag bit last_A {1,0} and none_A {0,0}; the to-be-sorted queue 1_B contains to-be-sorted data data_b {9}, sorting data flag bits last_b {1} and none_b {1};
(1) The data_A:5 corresponding flag bit none_A is detected to be low level, namely the queue 2_A to be ordered is valid data, and the data_B:9 corresponding flag bit none_B is detected to be high level, namely the queue 2_B to be ordered is invalid data;
(2) Reading an empty queue 1_B to be ordered, reading all data data_A {1,2,3,4,5} of the queue 1_A to be ordered according to the read sequence, writing the data into an ordering result queue, when the data_A 1 is read, setting last_flag to be high level when last_A is detected to be high level, and setting none_flag to be low level when data_A 1 is written into the ordering result queue;
(3) The merged ordered data is: {1,2,3,4,5}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0,0,0,0}.
Embodiment III
As shown in fig. 5, this embodiment takes the secondary merge ordering of 4 sets of data in the database as an example, and the queues to be ordered 1_a, 1_B, 2_A, 2_B are subjected to ascending merge ordering.
At the first stage of merging: the queue 1_A to be ordered comprises data data_A {1,2,3}, and an ordered data flag bit last_A {1,0} and none_A {0,0}; the to-be-sorted queue 1_B contains to-be-sorted data data_b {5}, sorting data flag bits last_b {1} and none_b {1};
(1) It is detected that none_a corresponding to data_a3 is low level, i.e. the to-be-ordered queue 1_a is valid data,
the none_B corresponding to the data_B:5 is at a high level, namely the queue 1_B to be ordered is invalid data;
(2) Reading an empty queue 1_B to be ordered, reading all data data_A {1,2,3} of the queue 1_A to be ordered according to the read sequence, writing the data_A {1,2,3} into an ordering result queue, and when the data_A 1 is read out, setting the last_A 1 into the ordering result queue as high level when the last_A is detected, and setting the none_flag as low level;
(3) The merged ordered data is: {1,2,3}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0,0}.
And (3) the same principle: for the first level merge sort: the to-be-sorted queue 2_A contains to-be-sorted data data_A {9}, sorting data flag bits last_A {1} and none_A {1}; the to-be-sorted queue 2_B contains to-be-sorted data data_b {7}, sorting data flag bits last_b {1} and none_b {1};
(1) The data_A:9 corresponding flag bit none_A is detected to be high level, namely the queue 2_A to be ordered is valid data, and the data_B:7 corresponding flag bit none_B is detected to be high level, namely the queue 2_B to be ordered is invalid data;
(2) Reading the empty queue 2_B to be ordered and the queue 2_A to be ordered; writing data_A {9} into the sequencing result queue, and setting last_flag to be high level and none_flag to be high level;
(3) The merged ordered data is: {9}, corresponding last_flag is: {1}, corresponding none_flag is: {1}.
For the second level merge sort: the queue 1_A to be ordered comprises data data_A {1,2,3}, and an ordered data flag bit last_A {1,0} and none_A {0,0}; the to-be-sorted queue 1_B contains to-be-sorted data data_b {9}, sorting data flag bits last_b {1} and none_b {1};
(1) The data_A:3 corresponding flag bit none_A is detected to be low level, namely the queue 2_A to be ordered is valid data, the data_B:9 corresponding flag bit none_B is detected to be high level, namely the queue 2_B to be ordered is invalid data;
(2) Reading an empty queue 1_B to be ordered, reading all data data_A {1,2,3} of the queue 1_A to be ordered according to the read sequence, writing the data_A {1,2,3} into an ordering result queue, and when the data_A 1 is read out, setting the last_A 1 into the ordering result queue as high level when the last_A is detected, and setting the none_flag as low level;
(3) The merged ordered data is: {1,2,3}, corresponding last_flag is: {1,0}, corresponding none_flag is: {0,0,0}.
The above embodiment describes 2-level data merging and arrangement with different lengths, the flow of the multi-level merging and arrangement method is similar to that of 2-level data merging and arrangement, and the implementation scheme and structure of each level 2 merging and ordering module are identical.
The foregoing describes in detail preferred embodiments of the present invention. It should be understood that numerous modifications and variations can be made in accordance with the concepts of the invention by one of ordinary skill in the art without undue burden. Therefore, all technical solutions which can be obtained by logic analysis, reasoning or limited experiments based on the prior art by the person skilled in the art according to the inventive concept shall be within the scope of protection defined by the claims.
Claims (5)
1. The multistage merging and sorting method based on the FPGA is characterized by comprising the following steps of:
step 1, respectively detecting a plurality of 2-merging and sorting modules, wherein each 2-merging and sorting module comprises a queue A to be sorted and a queue B to be sorted;
step 2, respectively carrying out merging and sorting on the plurality of 2 merging and sorting modules, and respectively outputting the arrangement results;
step 3, the arrangement result is used as a queue to be ordered of the next stage, a 2 merging and ordering module of the next stage is entered, and the steps 1 and 2 are repeated until a final arrangement result is output;
in the step 1, the first stage includes M2 merge sort modules, where the 2 merge sort modules include a to-be-sorted queue a and a to-be-sorted queue B, the to-be-sorted queue a and the to-be-sorted queue B of the 1 st stage are set to a first data pre-reading mode, the to-be-sorted queue a and the to-be-sorted queue B respectively store data queues that have been sorted according to a certain rule, and the to-be-sorted queue a and the to-be-sorted queue B can respectively output to-be-sorted data data_ A, data _b, a last flag bit last_ A, last _b of the to-be-sorted data, a flag bit none_ A, none _b of the to-be-sorted data and a flag bit fifo_empty_ A, fifo _empty_b of the to-be-sorted data;
the to-be-sequenced queue A in the 2-merge sequencing module comprises to-be-sequenced data data_A, and to-be-sequenced data flag bits last_A and none_A; the to-be-sorted queue B comprises to-be-sorted data data_B, a to-be-sorted data flag bit last_B and a none_B; the sequencing result queue comprises sequenced data, a sequenced data flag bit last_flag and a none_flag;
the merging and sorting method of each 2-merging and sorting module comprises the following steps:
step 2-1), detecting a data_A flag bit none_A to be ordered, wherein the data_B flag bit none_B level to be ordered, if none_A is low level and none_B is low level, jumping to step 2-2); if none_A is high level and none_B is low level, jumping to step 2-7); if none_A is low level and none_B is high level, jumping to step 2-8); if none_A is high level, none_B is high level, jumping to step 2-9);
step 2-2), detecting last_A and last_B, if last_A is high level, last_B is low level, and jumping to step 2-4); if last_A is low level, last_B is high level, jump to step 2-5); if last_A is high level, last_B is high level, jump to step 2-6); if last_A is low level, last_B is low level, jump to step 2-3);
step 2-3), comparing the data to be sorted data_A with the data to be sorted data_B, and jumping to step 2-2) if the data_A is arranged in front and is written into the sorting result queue according to the set sorting rule A, and reading the next data data_A of the sorting result queue A to continuously compare with the data_B; otherwise, writing the data_B into the sequencing result queue, reading the next data data_B of the to-be-sequenced queue B, continuously comparing the next data data_B with the data_A, and jumping to the step 2-2);
step 2-4), comparing the data to be sorted data_A with the data to be sorted data_B, if the data_A is arranged in front, writing the data_A into a sorting result queue according to a set sorting rule A, then writing all data of the queue to be sorted B into the sorting result queue, setting last_flag as high level and none_flag as low level when last_B is high level, and jumping to step 2-1); otherwise, writing the data_B into the sequencing result queue, reading the next data data_B of the queue B to be sequenced, continuing to compare, and jumping to the step 2-2);
step 2-5), comparing the data to be sorted data_A with the data to be sorted data_B, if the data_B is arranged in front, writing the data_B into a sorting result queue according to a set sorting rule A, then writing all data of the queue to be sorted A into the sorting result queue, setting last_flag as high level and none_flag as low level when last_A is high level, and jumping to step 2-1); otherwise, writing the data_A into an ordering result queue, reading the next data data_A of the queue A to be ordered, continuing to compare, and jumping to the step 2-2);
step 2-6), comparing the data to be sorted data_A with the data to be sorted data_B, and jumping to step 2-1) according to a set sorting rule A if the data_A is arranged in front and is written into a sorting result queue, then writing the data_B into the sorting result queue and setting last_flag to be high level and none_flag to be low level; otherwise, writing the data_B into the sequencing result queue, then writing the data_A into the sequencing result queue, setting last_flag to be high level, setting none_flag to be low level, and jumping to the step 2-1);
step 2-7), reading an empty queue A to be ordered, writing all data of the queue B to be ordered into an ordering result queue, setting last_flag to be high level when last_B is high level, setting none_flag to be low level, and jumping to step 2-1);
step 2-8), reading an empty queue B to be ordered, writing all data of the queue A to be ordered into an ordering result queue, setting last_flag to be high level when last_A is high level, setting none_flag to be low level, and jumping to step 2-1);
step 2-9), writing the data_A into the sequencing result queue, setting the last_flag to be high level, setting the none_flag to be high level, reading the empty queue A to be sequenced and the queue B to be sequenced, and jumping to the step 2-1).
2. The method for realizing multi-stage merging and sorting based on the FPGA as claimed in claim 1, wherein the operation of simultaneous merging and sorting of 2M input channels is realized by adopting a K-stage cascade mode, each stage comprises one or more 2 merging and sorting modules, and each 2 merging and sorting module of each stage adopts the same structure and realization method.
3. The method for multi-stage merge sort based on FPGA implementation as claimed in claim 2, wherein said to-be-sorted queue a and said to-be-sorted queue B in all M2-merge sort modules of the first stage must store at least one to-be-merged sort data; the to-be-ordered queue for each stage to 2-merge ordering must store any data with a last or none flag high.
4. The method for implementing multi-level merge ordering based on FPGA as claimed in claim 3, wherein, none_A is high level and none_B is low level, only data_B is written into the ordering result queue; the none_B is high level, the none_A is low level, and only the data_A is written into the sequencing result queue; both none_A and none_B are high, only data_A is written into the ordering result queue.
5. The method for multi-stage merge ordering implemented based on FPGA as recited in claim 3, wherein the data length of the queues a and B to be ordered in the 2-merge ordering module is arbitrary.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010391104.5A CN111562898B (en) | 2020-05-11 | 2020-05-11 | Multistage merging and sorting method based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010391104.5A CN111562898B (en) | 2020-05-11 | 2020-05-11 | Multistage merging and sorting method based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111562898A CN111562898A (en) | 2020-08-21 |
| CN111562898B true CN111562898B (en) | 2023-06-09 |
Family
ID=72068010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010391104.5A Active CN111562898B (en) | 2020-05-11 | 2020-05-11 | Multistage merging and sorting method based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111562898B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112947890B (en) * | 2021-03-09 | 2021-11-02 | 中科驭数（北京）科技有限公司 | Merging and sorting method and device |
| CN112947889A (en) * | 2021-03-09 | 2021-06-11 | 中科驭数（北京）科技有限公司 | Merging and sorting method and device |
| CN113076312B (en) * | 2021-03-11 | 2022-11-18 | 中科驭数（北京）科技有限公司 | Merging tree form sorting device, sorting system and sorting method |
| CN113177024B (en) * | 2021-06-29 | 2021-09-14 | 南京烽火星空通信发展有限公司 | Data global merging method under mass data scene |
| CN116342371B (en) * | 2023-03-24 | 2024-05-24 | 摩尔线程智能科技(北京)有限责任公司 | Method for GPU and secondary cache, GPU and secondary cache |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102968496A (en) * | 2012-12-04 | 2013-03-13 | 天津神舟通用数据技术有限公司 | Parallel sequencing method based on task derivation and double buffering mechanism |
| CN103226464A (en) * | 2013-03-29 | 2013-07-31 | 江苏复芯物联网科技有限公司 | Merging sort structure |
| CN104932864A (en) * | 2015-06-25 | 2015-09-23 | 许继电气股份有限公司 | Merging-sorting method based on assembly line process and valve control device using merging-sorting method |
| CN107402741A (en) * | 2017-08-04 | 2017-11-28 | 电子科技大学 | A kind of sort method for being suitable for FPGA realizations |
| CN107908714A (en) * | 2017-11-10 | 2018-04-13 | 上海达梦数据库有限公司 | A kind of aggregation of data sort method and device |
-
2020
- 2020-05-11 CN CN202010391104.5A patent/CN111562898B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102968496A (en) * | 2012-12-04 | 2013-03-13 | 天津神舟通用数据技术有限公司 | Parallel sequencing method based on task derivation and double buffering mechanism |
| CN103226464A (en) * | 2013-03-29 | 2013-07-31 | 江苏复芯物联网科技有限公司 | Merging sort structure |
| CN104932864A (en) * | 2015-06-25 | 2015-09-23 | 许继电气股份有限公司 | Merging-sorting method based on assembly line process and valve control device using merging-sorting method |
| CN107402741A (en) * | 2017-08-04 | 2017-11-28 | 电子科技大学 | A kind of sort method for being suitable for FPGA realizations |
| CN107908714A (en) * | 2017-11-10 | 2018-04-13 | 上海达梦数据库有限公司 | A kind of aggregation of data sort method and device |
Non-Patent Citations (7)
| Title |
|---|
| Ajitesh Srivastava等.A Hybrid Design for High Performance Large-scale Sorting on FPGA.《IEEE》.2015,第1-6页. * |
| Alexander Antonov等.Research of the Efficiency of High-level Synthesis Tool for FPGA Based Hardware Implementation of Some Basic Algorithms for the Big Data Analysis and Management Tasks.《PROCEEDING OF THE 26TH CONFERENCE OF FRUCT ASSOCIATION》.2020,第23-29页. * |
| Dirk Koch等.FPGASort: A High Performance Sorting Architecture Exploiting Run-time Reconfiguration on FPGAs for Large Problem Sorting.《ACM》.2011,第1-10页. * |
| Takuma Usui等.A Cost-Effective and Scalable Merge Sorter Tree on FPGAs.《2016 Fourth International Symposium on Computing and Networking》.2016,第47-56页. * |
| 宋宇飞等.基于IP协议的FPGA万兆可靠互联通信设计与实现.《上海大学学报（自然科学版）》.2015,第570-578页. * |
| 王莎等.基于FPGA的CAN总线通信*** .《工业控制计算机》.2018,第1-3页. * |
| 袁文燕等.FPGA片间万兆可靠通信的设计与实现.《电视技术》.2014,第43-46页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111562898A (en) | 2020-08-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111562898B (en) | Multistage merging and sorting method based on FPGA | |
| US10379816B2 (en) | Data accumulation apparatus and method, and digital signal processing device | |
| US11977902B2 (en) | Methods and systems for event reporting | |
| Guo et al. | Fast binary counters and compressors generated by sorting network | |
| CN111813370A (en) | Multi-path parallel merging and sorting system based on FPGA | |
| US20170193351A1 (en) | Methods and systems for vector length management | |
| Lipu et al. | Exploiting parallelism for faster implementation of Bubble sort algorithm using FPGA | |
| US20200401553A1 (en) | Devices for time division multiplexing of state machine engine signals | |
| CN112015366B (en) | Data sorting method, data sorting device and database system | |
| JPH08339291A (en) | Selection circuit of maximum value | |
| CA1286732C (en) | Rank-order filter | |
| US20030095718A1 (en) | Median filter | |
| KR100417908B1 (en) | A processing circuit and a search processor circuit | |
| CN109684761B (en) | Wide exclusive nor circuit optimization method | |
| CN110909793B (en) | Decision forest system and decision forest reasoning method based on FPGA | |
| Lin et al. | Two-dimensional rank-order filter by using max-min sorting network | |
| Choudhury et al. | Training accelerator for two means decision tree | |
| Sri et al. | An Efficient Quasi Comparison-free Bidirectional Architecture for Sorting Algorithm | |
| Dimitrakopoulos et al. | Sorter based permutation units for media-enhanced microprocessors | |
| Mumolo | VHDL design of a scalable VLSI sorting device based on pipelined computation | |
| JP3525960B2 (en) | Parallel sort method | |
| Santambrogio et al. | A new compact SD2 positive integer triangular array division circuit | |
| JP3447180B2 (en) | Data operation circuit | |
| Chen et al. | A bit-split byte-parallel string matching architecture | |
| Chen et al. | Fast scalable algorithm on LARPBS for sequence alignment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231128 Address after: 200444 zone B, floor 5, building 1, No. 668, SHANGDA Road, Baoshan District, Shanghai Patentee after: Shanghai Tianyu Optical Communication Technology Co.,Ltd. Address before: 200444 No. 99, upper road, Shanghai, Baoshan District Patentee before: Shanghai University |
|
| TR01 | Transfer of patent right |