CN113554574A - Compressed sensing image recovery method, device, equipment and medium - Google Patents
Compressed sensing image recovery method, device, equipment and medium Download PDFInfo
- Publication number
- CN113554574A CN113554574A CN202111110482.2A CN202111110482A CN113554574A CN 113554574 A CN113554574 A CN 113554574A CN 202111110482 A CN202111110482 A CN 202111110482A CN 113554574 A CN113554574 A CN 113554574A
- Authority
- CN
- China
- Prior art keywords
- matrix
- group
- compressed sensing
- operations
- calculation steps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011084 recovery Methods 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 51
- 239000011159 matrix material Substances 0.000 claims abstract description 343
- 238000004364 calculation method Methods 0.000 claims abstract description 130
- 238000012545 processing Methods 0.000 claims abstract description 58
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 49
- 238000005259 measurement Methods 0.000 claims abstract description 48
- 239000013598 vector Substances 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 18
- 238000000354 decomposition reaction Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 8
- 238000003384 imaging method Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Data Mining & Analysis (AREA)
- Computational Mathematics (AREA)
- Computing Systems (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Image Processing (AREA)
Abstract
The application discloses a compressed sensing image recovery method, a device, equipment and a medium, comprising the following steps: decomposing the measurement matrix in the compressed sensing recovery algorithm to obtain a plurality of small-size matrixes; respectively determining matrix operation between a sensing matrix and each small-size matrix in a compressed sensing recovery algorithm to obtain a plurality of groups of matrix operation; decomposing each group of matrix operation respectively to obtain all calculation steps corresponding to each group of matrix operation, and dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules; and processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability, and merging the operation results of each group to obtain a compressed sensing image recovery result. By splitting and decoupling the large-size matrix operation in the compressed sensing and executing the module obtained after decoupling on a processing chip with parallel operation capability, the operation time can be shortened and the image recovery speed can be improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for compressed sensing image restoration.
Background
At present, due to the advantages of high detection efficiency, low noise, low cost, simple structure and the like, Compressed Sensing (CS) has a great application prospect in the fields of space remote Sensing, optical encryption transmission, medical imaging and the like. The compressed sensing imaging technology is a novel imaging technology for recovering scene information to be detected by utilizing the light field intensity correlation characteristic. Different from the direct imaging of the traditional area array detector, the compressed sensing only utilizes a single-point detector without spatial resolution, and the reconstruction of the target scene information can be realized by correlating the fluctuation of the light field and the total light intensity change of the echo.
The compressed sensing imaging process mainly comprises two independent modules of data acquisition and image recovery, wherein the image recovery module is usually the module which consumes the longest time in the whole compressed sensing imaging process, and the current compressed sensing image recovery process mainly has the following three problems: firstly, the data size of the matrix operation is large and all the matrix operations are executed in series; secondly, the operation process is complex and the operation steps are multiple, and thirdly, the data operation intermediate results are seriously depended.
The existing compressed sensing image restoration technology has the problems, so that the imaging time is too long, large-picture imaging is difficult to realize, and the actual application of compressed sensing is not facilitated. Therefore, how to shorten the time of matrix operation, reduce the complexity of the operation process, and remove the dependency of intermediate results of data operation is a problem to be solved in the art.
Disclosure of Invention
In view of the above, an object of the present application is to provide a method, an apparatus, a device and a medium for recovering a compressed sensing image, which can greatly shorten a matrix operation time and improve a recovery speed of the compressed sensing image. The specific scheme is as follows:
in a first aspect, the present application discloses a compressed sensing image restoration method, including:
decomposing a measurement matrix in a compressed sensing recovery algorithm to obtain a plurality of small-size matrixes;
respectively determining matrix operation between a sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operation;
decomposing each group of matrix operation respectively to obtain all calculation steps corresponding to each group of matrix operation, and dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain an operation module set corresponding to each group of matrix operation;
processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain an operation result of each group of matrix operation, and merging the operation results of each group of matrix operation to obtain a compressed sensing image recovery result.
Optionally, decomposing the measurement matrix in the compressed sensing recovery algorithm to obtain a plurality of small-size matrices includes:
decomposing a measurement matrix in a compressed sensing recovery algorithm by columns to obtain a plurality of column vectors with the same number as the columns of the measurement matrix;
correspondingly, the determining matrix operations between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix respectively to obtain a plurality of groups of matrix operations includes:
and respectively determining matrix operation between a sensing matrix in the compressed sensing recovery algorithm and each column vector to obtain a plurality of groups of matrix operation.
Optionally, the decomposing each group of the matrix operations to obtain all calculation steps corresponding to each group of the matrix operations respectively includes:
respectively determining all operation functions contained in each group of matrix operation;
and respectively decomposing each group of matrix operation based on a decomposition mode that one operation function corresponds to one calculation step to obtain all calculation steps corresponding to each group of matrix operation.
Optionally, the dividing different calculation steps in which data dependency exists in each group of matrix operations into different operation modules to obtain an operation module set corresponding to each group of matrix operations includes:
determining the dependency relationship of input and output data between each calculation step;
and based on the dependency relationship of the input and output data, dividing different calculation steps in each group of matrix operation respectively so as to divide the different calculation steps with the dependency relationship into different operation modules to obtain an operation module set corresponding to each group of matrix operation.
Optionally, the processing chip with parallel operation capability processes the operation module set corresponding to each group of matrix operations to obtain an operation result of each group of matrix operations, including:
determining the operation priority of each operation module in the operation module set based on the data dependency among different calculation steps;
and processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability according to the operation priority to obtain an operation result of each group of matrix operation.
Optionally, the merging the operation results of each group of matrix operations to obtain a compressed sensing image recovery result includes:
and sequentially combining the operation results of each group of matrix operation according to the position sequence of the small-size matrix corresponding to the matrix operation in the measurement matrix to obtain a compressed sensing image recovery result.
Optionally, the processing chip with parallel operation capability processes the operation module set corresponding to each group of matrix operations to obtain an operation result of each group of matrix operations, including:
and loading the operation module set corresponding to each group of matrix operation onto a chip of a field programmable logic gate array chip for storage, and performing running parallel processing on the operation module set corresponding to each group of matrix operation by using the field programmable logic gate array chip to obtain an operation result of each group of matrix operation.
In a second aspect, the present application discloses a compressed sensing image restoration apparatus, comprising:
the matrix decomposition module is used for decomposing the measurement matrix in the compressed sensing recovery algorithm to obtain a plurality of small-size matrixes;
the operation determining module is used for respectively determining matrix operation between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operation;
the step division module is used for decomposing each group of matrix operation respectively to obtain all calculation steps corresponding to each group of matrix operation, and dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain an operation module set corresponding to each group of matrix operation;
and the merging module is used for processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain the operation result of each group of matrix operation, and merging the operation results of each group of matrix operation to obtain a compressed sensing image recovery result.
In a third aspect, the present application discloses an electronic device comprising a processor and a memory; wherein the processor implements the aforementioned compressed sensing image restoration method when executing the computer program stored in the memory.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the aforementioned compressed sensing image restoration method.
It can be seen that the present application first decomposes the measurement matrix in the compressed sensing recovery algorithm to obtain a plurality of small-size matrices, then respectively determining matrix operation between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operations, respectively decomposing each group of matrix operations to obtain all calculation steps corresponding to each group of matrix operations, dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain operation module sets corresponding to each group of matrix operation, and finally processing the operation module sets corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain operation results of each group of matrix operation, and combining the operation results of each group of matrix operation to obtain a compressed sensing image recovery result. Therefore, the method and the device have the advantages that the large-size measurement matrix in the compressed sensing is split to obtain a plurality of small-size operations, the matrix operations between the small-size matrix and the sensing matrix are refined and decomposed to obtain a plurality of calculation steps, the calculation steps are divided into different operation modules according to the dependency relationship between the data of the calculation steps, and the calculation steps in the operation modules are executed in parallel on a processing chip with parallel operation capability, so that the matrix operation time can be greatly shortened, and the speed of recovering the compressed sensing image is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart of a compressed sensing image restoration method disclosed in the present application;
FIG. 2 is a flowchart of a specific compressed sensing image restoration method disclosed in the present application;
FIG. 3 is a diagram illustrating a split view of a compressed sensing measurement matrix according to an embodiment of the present disclosure;
FIG. 4 is an exploded view of a specific small-scale matrix operation disclosed herein;
FIG. 5 is a schematic diagram of a specific module division disclosed herein;
FIG. 6 is a schematic diagram illustrating a pipeline parallel execution of a specific FPGA according to the present disclosure;
FIG. 7 is a diagram illustrating a specific combination of results of small-size matrix operations disclosed herein;
FIG. 8 is a schematic diagram of a compressed sensing image restoration apparatus according to the present disclosure;
fig. 9 is a block diagram of an electronic device disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application discloses a compressed sensing image recovery method, and as shown in fig. 1, the method comprises the following steps:
step S11: the measurement matrix in the compressed sensing recovery algorithm is decomposed to obtain a plurality of small-size matrices.
In this embodiment, the sensing matrix in the compressed sensing recovery algorithm that does not change due to the image difference is kept unchanged, and the measurement matrix in the compressed sensing recovery algorithm that differs due to the image difference is split to obtain a plurality of corresponding small-size matrices. It should be noted that when the measurement matrix is split, the measurement matrix may be split according to an actual application situation, the measurement matrix may be split by columns, or may be split by rows, and when the measurement matrix is split by rows or columns, the measurement matrix may be split into a single column or a single row, or may be split into multiple columns or multiple rows.
Step S12: and respectively determining matrix operation between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operation.
In this embodiment, after decomposing a measurement matrix in a compressed sensing recovery algorithm to obtain a plurality of small-size matrices, a type of matrix operation between the sensing matrix and each of the small-size matrices may be determined, and further, the determined type of matrix operation may be used to perform matrix operation on the sensing matrix and each of the small-size matrices to obtain a plurality of sets of matrix operations having the same number as the small-size matrices.
Step S13: and decomposing each group of matrix operation respectively to obtain all calculation steps corresponding to each group of matrix operation, and dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain an operation module set corresponding to each group of matrix operation.
In this embodiment, after determining the matrix operation between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix respectively, and obtaining a plurality of sets of the matrix operations, the operation process of each set of the matrix operations is refined and decomposed respectively, and the calculation step corresponding to each set of the matrix operations is refined and expanded, after the calculation step is obtained, the relationship between the data in the calculation step corresponding to each set of the matrix operations is analyzed, and according to the correlation of the data in the operation process, whether a data dependency relationship exists between different calculation steps in each set is determined, wherein the data dependency relationship includes, but is not limited to, a dependency relationship between data input and output, if a data dependency relationship exists between the different calculation steps, the different calculation steps having the data dependency relationship are divided into different calculation modules, and performing the division processing on the calculation steps in all the matrix operations of each group to obtain an operation module set corresponding to the matrix operations of each group.
Step S14: processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain an operation result of each group of matrix operation, and merging the operation results of each group of matrix operation to obtain a compressed sensing image recovery result.
In this embodiment, after decomposing each set of matrix operations to obtain all the calculation steps corresponding to each set of matrix operations, dividing the different calculation steps having data dependency relationship in each set of matrix operations into different calculation modules to obtain a set of calculation modules corresponding to each set of matrix operations, transplanting the different calculation modules having no data dependency relationship after being split in all the sets of matrix operations to a processing chip having parallel calculation capability, and performing the calculation steps included in the calculation modules in a pipelined parallel manner on the processing chip having parallel calculation capability to obtain the calculation results of each corresponding set of matrix operations, after obtaining the calculation results of each set of matrix operations, merging the calculation results of the matrix operations, and obtaining the result after merging, restoration of the compressed perceptual image may be achieved.
In this embodiment, the merging the operation results of each group of matrix operations to obtain a compressed sensing image recovery result specifically may include: and sequentially combining the operation results of each group of matrix operation according to the position sequence of the small-size matrix corresponding to the matrix operation in the measurement matrix to obtain a compressed sensing image recovery result. It can be understood that after the operation results of each group of matrix operations are obtained, the operation results of each group of matrix operations may be sequentially merged according to the sequence of the positions of the small-size matrices corresponding to the matrix operations in the compressed sensing recovery algorithm, and the merged results are used as the compressed sensing image recovery results.
It can be seen that, in the embodiment of the present application, the measurement matrix in the compressed sensing recovery algorithm is decomposed to obtain a plurality of small-size matrices, then respectively determining matrix operation between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operations, respectively decomposing each group of matrix operations to obtain all calculation steps corresponding to each group of matrix operations, dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain operation module sets corresponding to each group of matrix operation, and finally processing the operation module sets corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain operation results of each group of matrix operation, and combining the operation results of each group of matrix operation to obtain a compressed sensing image recovery result. Therefore, the embodiment of the application obtains a plurality of small-size operations by splitting the large-size measurement matrix in the compressed sensing, and performs the detailed decomposition on the matrix operations between the small-size matrix and the sensing matrix to obtain a plurality of calculation steps, divides the calculation steps into different operation modules according to the dependency relationship between the data of the calculation steps, and executes the calculation steps in the operation modules in parallel on the processing chip with the parallel operation capability, so that the matrix operation time can be greatly shortened, and the speed of recovering the compressed sensing image can be improved.
The embodiment of the application discloses a specific compressed sensing image recovery method, which is shown in fig. 2 and comprises the following steps:
step S21: and decomposing the measurement matrix in the compressed sensing recovery algorithm by columns to obtain a plurality of column vectors with the same number of columns as the measurement matrix.
In this embodiment, when decomposing the measurement matrix in the compressed sensing recovery algorithm, the sensing matrix in the compressed sensing recovery algorithm that does not change due to the image difference is kept unchanged, and the measurement matrix in the compressed sensing recovery algorithm that differs due to the image difference is split by columns to obtain a plurality of column vectors having the same number of columns as the measurement matrix. Specifically, as shown in fig. 3, a sensing matrix a having a size of M × N is kept unchanged, and a measurement matrix B having a size of N × K, which varies from one image to another, is divided into columns to obtain a column vector BX having a size of K columns of N × 1 (X is 1 ≦ X ≦ K).
Step S22: and respectively determining matrix operation between a sensing matrix in the compressed sensing recovery algorithm and each column vector to obtain a plurality of groups of matrix operation.
In this embodiment, after the measurement matrix in the compressed sensing recovery algorithm is decomposed by columns to obtain a plurality of column vectors with the same number as the columns of the measurement matrix, the types of matrix operations between the sensing matrix in the compressed sensing recovery algorithm and each of the column vectors may be further determined, and a plurality of sets of the matrix operations with the same number as the columns of the measurement matrix may be obtained. For example, as shown in fig. 3, after splitting a measurement matrix B with a size of N × K by columns to obtain K column vectors BX with a size of N × 1 (1 ≦ BX ≦ K), matrix operations may be performed between a sensing matrix a in the compressed sensing recovery algorithm and each column vector BX, respectively, to obtain K sets of the matrix operations.
Step S23: and respectively determining all operation functions contained in each group of matrix operation.
In this embodiment, after matrix operations between a sensing matrix in the compressed sensing recovery algorithm and each column vector are respectively determined to obtain a plurality of groups of matrix operations, an operation process of each group of matrix operations is expanded, and further, according to the expanded operation process, all operation functions included in each group of matrix operations are determined.
Step S24: and respectively decomposing each group of matrix operation based on a decomposition mode that one operation function corresponds to one calculation step to obtain all calculation steps corresponding to each group of matrix operation.
In this embodiment, after all the operation functions included in each group of matrix operations are determined, the operation process of each group of matrix operations may be decomposed according to the operation functions, and specifically, each group of matrix operations may be decomposed into a plurality of independent calculation steps with a single function based on a decomposition manner in which one operation function corresponds to one calculation step. For example, referring to fig. 4, in fig. 4, matrix operation is performed on a sensing matrix a and any column vector BX of a measurement matrix B split by columns, and an operation process of the matrix operation is decomposed based on a decomposition mode that one operation function corresponds to one calculation step, so as to obtain a single-function calculation step with a small granularity, such as step 1, step 2, and step W.
Step S25: and determining the dependency relationship of the input and output data between each calculation step.
In this embodiment, after each group of matrix operations is decomposed based on a decomposition mode in which one operation function corresponds to one calculation step, and all calculation steps corresponding to each group of matrix operations are obtained, different calculation steps are analyzed, and a dependency relationship between input and output data between each calculation step is determined. Specifically, if the input data and/or the output data in any one calculation step are used as the input data and/or the output data in another calculation step, it can be determined that the input data and the output data in the two calculation steps have a dependency relationship. For example, referring to fig. 4, if step 4 in fig. 4 is obtained from step 1, it can be determined that there is a dependency relationship between step 1 and step 4.
Step S26: and based on the dependency relationship of the input and output data, dividing different calculation steps in each group of matrix operation respectively so as to divide the different calculation steps with the dependency relationship into different operation modules to obtain an operation module set corresponding to each group of matrix operation.
In this embodiment, after determining the dependency relationship of the input/output data between each calculation step, different calculation steps in each set of the matrix operation may be divided based on the determined dependency relationship of the input/output data. Specifically, if it is determined that the dependency relationship of the input and output data exists between different calculation steps, it is indicated that there is a correlation between the input data and/or the output data of one of the different calculation steps and the input data and/or the output data of another one of the different calculation steps, the calculation steps having the dependency relationship of the input and output data are divided into different operation modules, and then an operation module set corresponding to each group of matrix operations is obtained. For example, referring to fig. 5, the calculation steps with data dependency relationship in fig. 4 are divided into different operation modules, and the calculation steps without data dependency relationship are divided into the same operation module. If the input and output data of the calculation step 1 and the input and output data of the calculation step 2 have no dependency relationship, dividing the calculation step 1 and the calculation step 2 into the same operation module, namely, into the module 1; and (3) if the input data of the calculation step 4 is the output data of the calculation step 1, dividing the calculation step 1 and the calculation step 4 into different operation modules, namely respectively dividing the operation modules into a module 1 and a module 2.
Step S27: and determining the operation priority of each operation module in the operation module set based on the data dependency relationship among different calculation steps.
In this embodiment, based on the dependency relationship of the input and output data, different calculation steps in each group of the matrix operations are respectively divided, so that different calculation steps with the dependency relationship are divided into different operation modules, and after an operation module set corresponding to each group of the matrix operations is obtained, the operation priority of each operation module in the operation module set can be further determined according to the sequence of the data dependency relationship among the different calculation steps. For example, as shown in fig. 5, after the calculation step 1 and the calculation step 4, which have the dependency relationship of the input and output data, are respectively divided into two different calculation modules, since the output of the calculation step 1 is the input of the calculation step 4, the calculation module in which the calculation step 1 is located is executed in preference to the calculation module in which the calculation step 4 is located.
Step S28: and processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability according to the operation priority, and merging operation results of each group of matrix operation to obtain a compressed sensing image recovery result.
In this embodiment, after determining the operation priority of each operation module in the operation module set based on the data dependency between different calculation steps, the operation module set corresponding to each group of matrix operations may be transplanted to a processing chip with parallel operation capability, and the calculation steps corresponding to all the operation modules in the operation module set are executed on the processing chip with parallel operation capability according to the operation priorities, so as to obtain the operation results of each group of matrix operations, and the operation results of each group of matrix operations are sequentially merged according to the sequence of the positions of the column vectors corresponding to the matrix operations in the measurement matrix, and the merged results are used as the compressed sensing image recovery results.
In a specific embodiment, the processing, by a processing chip with parallel operation capability, the processing the operation module set corresponding to each group of the matrix operations to obtain an operation result of each group of the matrix operations may include: and loading the operation module set corresponding to each group of matrix operation onto a chip of an FPGA (Field Programmable Gate Array) chip for storage, so that the FPGA chip is utilized to perform running parallel processing on the operation module set corresponding to each group of matrix operation, and an operation result of each group of matrix operation is obtained. It can be understood that after the operation module set is obtained and the operation priority of each operation module in the operation module set is determined, each operation module in the operation module set may be transplanted to an FPGA chip for on-chip storage, and the computation steps in the number operation modules are subjected to pipeline parallel processing by using the powerful parallel processing characteristic of the FPGA chip, so as to obtain the operation result of each group of matrix operations. Specifically, referring to fig. 6, after the dependency relationship is removed, each operation module in fig. 5 is transplanted to a cycle unit of the FPGA chip and is subjected to parallel processing, for example, in fig. 6, the first cycle unit executes step 1 of module 1, the second cycle unit executes step 2 of module 1 and step 4 of module 2 in parallel, the third cycle unit executes step 3 of module 1, step 5 of module 2 and step 7 of module 3 in parallel, the fourth cycle unit executes step 6 of module 2 and step 8 of module 3 simultaneously, and the fifth cycle unit executes step 9 of module 3, and after the execution, an operation result Rx of the whole small-size matrix is obtained (1 ≦ X ≦ K). Further, all the operation matrixes are subjected to pipeline parallel processing on the FPGA chip, and as shown in FIG. 7, the operation matrixes are combined according to the operation priority of each matrix operation in the operation module set, so that a final combination result R with the size of M multiplied by K is obtained, and the matrix R is a compressed sensing image recovery result.
It can be seen that, in the embodiment of the present application, a measurement matrix in a compressed sensing recovery algorithm is first decomposed by columns to obtain a plurality of column vectors with the same number as that of the measurement matrix, then matrix operations between the sensing matrix in the compressed sensing recovery algorithm and each column vector are respectively determined to obtain a plurality of groups of matrix operations, all operation functions included in each group of matrix operations are respectively determined, then each group of matrix operations is decomposed based on a decomposition mode that one operation function corresponds to one calculation step to obtain all calculation steps corresponding to each group of matrix operations, a dependency relationship of input and output data between each calculation step is determined, then different calculation steps in each group of matrix operations are respectively divided based on the dependency relationship of the input and output data to divide different calculation steps having the dependency relationship into different operation modules, and finally, determining the operation priority of each operation module in the operation module set based on the data dependency among different calculation steps, processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability according to the operation priority, and combining the operation results of each group of matrix operation to obtain a compressed sensing image recovery result. It can be seen that in the embodiment of the application, a large-size measurement matrix in compressed sensing is split according to columns to obtain a plurality of independent small-size column vectors with single functions, the column vectors are respectively subjected to matrix operation with a sensing matrix in a compressed sensing recovery algorithm, then the matrix operation is subjected to refinement and decomposition to obtain a plurality of calculation steps, the calculation steps are divided into different operation modules according to the dependency relationship among the data of the calculation steps, and then the calculation steps in the operation modules are executed in parallel on a processing chip with parallel operation capability, so that the matrix operation time can be greatly shortened, and the speed of compressed sensing image recovery is improved.
Correspondingly, the embodiment of the present application further discloses a compressed sensing image restoration apparatus, as shown in fig. 8, the apparatus includes:
the matrix decomposition module 11 is configured to decompose a measurement matrix in a compressed sensing recovery algorithm to obtain a plurality of small-size matrices;
an operation determining module 12, configured to determine matrix operations between the sensing matrices in the compressed sensing recovery algorithm and each of the small-size matrices, respectively, to obtain multiple sets of the matrix operations;
a step dividing module 13, configured to decompose each group of matrix operations to obtain all calculation steps corresponding to each group of matrix operations, and divide different calculation steps having a data dependency relationship in each group of matrix operations into different operation modules to obtain an operation module set corresponding to each group of matrix operations;
the merging module 14 is configured to process the operation module set corresponding to each group of matrix operations through a processing chip with parallel operation capability to obtain an operation result of each group of matrix operations, and merge the operation results of each group of matrix operations to obtain a compressed sensing image recovery result.
For the specific work flow of each module, reference may be made to corresponding content disclosed in the foregoing embodiments, and details are not repeated here.
It can be seen that, in the embodiment of the present application, the measurement matrix in the compressed sensing recovery algorithm is decomposed to obtain a plurality of small-size matrices, then respectively determining matrix operation between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operations, respectively decomposing each group of matrix operations to obtain all calculation steps corresponding to each group of matrix operations, dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain operation module sets corresponding to each group of matrix operation, and finally processing the operation module sets corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain operation results of each group of matrix operation, and combining the operation results of each group of matrix operation to obtain a compressed sensing image recovery result. Therefore, the embodiment of the application obtains a plurality of small-size operations by splitting the large-size measurement matrix in the compressed sensing, and performs the detailed decomposition on the matrix operations between the small-size matrix and the sensing matrix to obtain a plurality of calculation steps, divides the calculation steps into different operation modules according to the dependency relationship between the data of the calculation steps, and executes the calculation steps in the operation modules in parallel on the processing chip with the parallel operation capability, so that the matrix operation time can be greatly shortened, and the speed of recovering the compressed sensing image can be improved.
In some specific embodiments, the matrix decomposition module 11 may specifically include:
the first matrix decomposition unit is used for decomposing a measurement matrix in a compressed sensing recovery algorithm by columns to obtain a plurality of column vectors with the same number as that of the columns of the measurement matrix;
correspondingly, the operation determining module 12 may specifically include:
and the operation determining unit is used for respectively determining matrix operation between a sensing matrix in the compressed sensing recovery algorithm and each column vector to obtain a plurality of groups of matrix operation.
In some specific embodiments, the step dividing module 13 may specifically include:
a first determining unit configured to determine all operation functions included in each of the sets of matrix operations, respectively;
and the operation decomposition unit is used for decomposing each group of matrix operation respectively based on a decomposition mode that one operation function corresponds to one calculation step so as to obtain all calculation steps corresponding to each group of matrix operation.
In some specific embodiments, the step dividing module 13 may specifically include:
a second determination unit for determining the dependency relationship of the input and output data between each calculation step;
the step dividing unit is used for dividing different calculation steps in each group of matrix operation respectively based on the dependency relationship of the input and output data so as to divide the different calculation steps with the dependency relationship to different operation modules to obtain an operation module set corresponding to each group of matrix operation.
In some specific embodiments, the merging module 14 may specifically include:
the priority determining unit is used for determining the operation priority of each operation module in the operation module set based on the data dependency relationship among different calculation steps;
and the module set processing unit is used for processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability according to the operation priority to obtain an operation result of each group of matrix operation.
In some specific embodiments, the merging module 14 may specifically include:
and the merging unit is used for sequentially merging the operation results of each group of matrix operation according to the position sequence of the small-size matrix corresponding to the matrix operation in the measurement matrix to obtain a compressed sensing image recovery result.
In some specific embodiments, the merging module 14 may specifically include:
and the loading unit is used for loading the operation module sets corresponding to each group of matrix operation to an FPGA chip for storage, so that the FPGA chip is utilized to perform running parallel processing on the operation module sets corresponding to each group of matrix operation to obtain the operation result of each group of matrix operation.
Further, an electronic device is disclosed in the embodiments of the present application, and fig. 9 is a block diagram of an electronic device 20 according to an exemplary embodiment, which should not be construed as limiting the scope of the application.
Fig. 9 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein, the memory 22 is used for storing a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the compressed sensing image restoration method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be specifically an electronic computer.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the storage 22 is used as a carrier for resource storage, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage manner may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device on the electronic device 20 and the computer program 222, and may be Windows Server, Netware, Unix, Linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the compressed sensing image restoration method performed by the electronic device 20 disclosed in any of the foregoing embodiments.
Further, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the compressed sensing image restoration method disclosed above. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The compressed sensing image restoration method, device, apparatus and medium provided by the present application are introduced in detail above, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A method for compressed sensing image restoration, comprising:
decomposing a measurement matrix in a compressed sensing recovery algorithm to obtain a plurality of small-size matrixes;
respectively determining matrix operation between a sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operation;
decomposing each group of matrix operation respectively to obtain all calculation steps corresponding to each group of matrix operation, and dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain an operation module set corresponding to each group of matrix operation;
processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain an operation result of each group of matrix operation, and merging the operation results of each group of matrix operation to obtain a compressed sensing image recovery result.
2. The method for compressed sensing image restoration according to claim 1, wherein decomposing the measurement matrix in the compressed sensing restoration algorithm to obtain a plurality of small-size matrices comprises:
decomposing a measurement matrix in a compressed sensing recovery algorithm by columns to obtain a plurality of column vectors with the same number as the columns of the measurement matrix;
correspondingly, the determining matrix operations between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix respectively to obtain a plurality of groups of matrix operations includes:
and respectively determining matrix operation between a sensing matrix in the compressed sensing recovery algorithm and each column vector to obtain a plurality of groups of matrix operation.
3. The method for restoring a compressed sensing image according to claim 1, wherein said decomposing each set of said matrix operations to obtain all computation steps corresponding to each set of said matrix operations respectively comprises:
respectively determining all operation functions contained in each group of matrix operation;
and respectively decomposing each group of matrix operation based on a decomposition mode that one operation function corresponds to one calculation step to obtain all calculation steps corresponding to each group of matrix operation.
4. The method for restoring a compressed sensing image according to claim 1, wherein the step of dividing different calculation steps having data dependency relationships in each group of matrix operations into different operation modules to obtain an operation module set corresponding to each group of matrix operations comprises:
determining the dependency relationship of input and output data between each calculation step;
and based on the dependency relationship of the input and output data, dividing different calculation steps in each group of matrix operation respectively so as to divide the different calculation steps with the dependency relationship into different operation modules to obtain an operation module set corresponding to each group of matrix operation.
5. The method for restoring a compressed sensing image according to claim 1, wherein the processing the operation module set corresponding to each group of matrix operations by a processing chip with parallel operation capability to obtain an operation result of each group of matrix operations comprises:
determining the operation priority of each operation module in the operation module set based on the data dependency among different calculation steps;
and processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability according to the operation priority to obtain an operation result of each group of matrix operation.
6. The method for restoring a compressed sensing image according to claim 1, wherein the combining the operation results of each group of the matrix operations to obtain the compressed sensing image restoration result comprises:
and sequentially combining the operation results of each group of matrix operation according to the position sequence of the small-size matrix corresponding to the matrix operation in the measurement matrix to obtain a compressed sensing image recovery result.
7. The compressed sensing image restoration method according to any one of claims 1 to 6, wherein the processing the operation module set corresponding to each group of matrix operations by a processing chip with parallel operation capability to obtain an operation result of each group of matrix operations includes:
and loading the operation module set corresponding to each group of matrix operation onto a chip of a field programmable logic gate array chip for storage, and performing running parallel processing on the operation module set corresponding to each group of matrix operation by using the field programmable logic gate array chip to obtain an operation result of each group of matrix operation.
8. A compressed sensing image restoration apparatus, comprising:
the matrix decomposition module is used for decomposing the measurement matrix in the compressed sensing recovery algorithm to obtain a plurality of small-size matrixes;
the operation determining module is used for respectively determining matrix operation between the sensing matrix in the compressed sensing recovery algorithm and each small-size matrix to obtain a plurality of groups of matrix operation;
the step division module is used for decomposing each group of matrix operation respectively to obtain all calculation steps corresponding to each group of matrix operation, and dividing different calculation steps with data dependency relationship in each group of matrix operation into different operation modules to obtain an operation module set corresponding to each group of matrix operation;
and the merging module is used for processing the operation module set corresponding to each group of matrix operation through a processing chip with parallel operation capability to obtain the operation result of each group of matrix operation, and merging the operation results of each group of matrix operation to obtain a compressed sensing image recovery result.
9. An electronic device comprising a processor and a memory; wherein the processor, when executing the computer program stored in the memory, implements the compressed perceptual image restoration method as defined in any one of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the compressed perceptual image restoration method as defined in any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111110482.2A CN113554574A (en) | 2021-09-23 | 2021-09-23 | Compressed sensing image recovery method, device, equipment and medium |
| PCT/CN2022/078489 WO2023045257A1 (en) | 2021-09-23 | 2022-02-28 | Compressed sensing image recovery method and apparatus, and device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111110482.2A CN113554574A (en) | 2021-09-23 | 2021-09-23 | Compressed sensing image recovery method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113554574A true CN113554574A (en) | 2021-10-26 |
Family
ID=78106502
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111110482.2A Pending CN113554574A (en) | 2021-09-23 | 2021-09-23 | Compressed sensing image recovery method, device, equipment and medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113554574A (en) |
| WO (1) | WO2023045257A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114022580A (en) * | 2022-01-06 | 2022-02-08 | 苏州浪潮智能科技有限公司 | Data processing method, device, equipment and storage medium for image compression |
| WO2023045257A1 (en) * | 2021-09-23 | 2023-03-30 | 苏州浪潮智能科技有限公司 | Compressed sensing image recovery method and apparatus, and device and medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090278539A1 (en) * | 2008-05-06 | 2009-11-12 | Philip James Beatty | System and method for using parallel imaging with compressed sensing |
| CN108199714A (en) * | 2017-12-22 | 2018-06-22 | 北京工业大学 | A kind of circuit system of improvement OMP algorithms restored applied to AIC architecture signals |
| CN108280818A (en) * | 2018-01-19 | 2018-07-13 | 清华大学深圳研究生院 | A kind of fast target imaging method and system based on compressed sensing |
| CN109194959A (en) * | 2018-09-28 | 2019-01-11 | 中国科学院长春光学精密机械与物理研究所 | A kind of compressed sensing imaging method, device, equipment, system and storage medium |
| CN110148139A (en) * | 2019-04-25 | 2019-08-20 | 电信科学技术研究院有限公司 | A kind of image recovery method and device |
| CN110659070A (en) * | 2018-06-29 | 2020-01-07 | 赛灵思公司 | High-parallelism computing system and instruction scheduling method thereof |
| CN112231630A (en) * | 2020-10-26 | 2021-01-15 | 国家超级计算无锡中心 | Sparse matrix solving method based on FPGA parallel acceleration |
| CN113365014A (en) * | 2021-05-11 | 2021-09-07 | 中国科学院国家空间科学中心 | Parallel compressed sensing GPU (graphics processing Unit) acceleration real-time imaging system and method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8788556B2 (en) * | 2011-05-12 | 2014-07-22 | Microsoft Corporation | Matrix computation framework |
| CN104537278A (en) * | 2014-12-01 | 2015-04-22 | 中国人民解放军海军工程大学 | Hardware acceleration method for predication of RNA second-stage structure with pseudoknot |
| WO2018094087A1 (en) * | 2016-11-17 | 2018-05-24 | The Mathworks, Inc. | Systems and methods for generating code for parallel processing units |
| CN113554574A (en) * | 2021-09-23 | 2021-10-26 | 苏州浪潮智能科技有限公司 | Compressed sensing image recovery method, device, equipment and medium |
-
2021
- 2021-09-23 CN CN202111110482.2A patent/CN113554574A/en active Pending
-
2022
- 2022-02-28 WO PCT/CN2022/078489 patent/WO2023045257A1/en unknown
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090278539A1 (en) * | 2008-05-06 | 2009-11-12 | Philip James Beatty | System and method for using parallel imaging with compressed sensing |
| CN108199714A (en) * | 2017-12-22 | 2018-06-22 | 北京工业大学 | A kind of circuit system of improvement OMP algorithms restored applied to AIC architecture signals |
| CN108280818A (en) * | 2018-01-19 | 2018-07-13 | 清华大学深圳研究生院 | A kind of fast target imaging method and system based on compressed sensing |
| CN110659070A (en) * | 2018-06-29 | 2020-01-07 | 赛灵思公司 | High-parallelism computing system and instruction scheduling method thereof |
| CN109194959A (en) * | 2018-09-28 | 2019-01-11 | 中国科学院长春光学精密机械与物理研究所 | A kind of compressed sensing imaging method, device, equipment, system and storage medium |
| CN110148139A (en) * | 2019-04-25 | 2019-08-20 | 电信科学技术研究院有限公司 | A kind of image recovery method and device |
| CN112231630A (en) * | 2020-10-26 | 2021-01-15 | 国家超级计算无锡中心 | Sparse matrix solving method based on FPGA parallel acceleration |
| CN113365014A (en) * | 2021-05-11 | 2021-09-07 | 中国科学院国家空间科学中心 | Parallel compressed sensing GPU (graphics processing Unit) acceleration real-time imaging system and method |
Non-Patent Citations (3)
| Title |
|---|
| 苗壮 等，: "采用GPU加速的压缩感知图像恢复算法", 《微电子学与计算机》 * |
| 董蕾 等，: "基于CUDA的压缩感知重构算法并行化研究", 《信息技术》 * |
| 陈帅 等，: "SAR图像压缩采样恢复的GPU并行实现", 《电子与信息学报》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023045257A1 (en) * | 2021-09-23 | 2023-03-30 | 苏州浪潮智能科技有限公司 | Compressed sensing image recovery method and apparatus, and device and medium |
| CN114022580A (en) * | 2022-01-06 | 2022-02-08 | 苏州浪潮智能科技有限公司 | Data processing method, device, equipment and storage medium for image compression |
| CN114022580B (en) * | 2022-01-06 | 2022-04-19 | 苏州浪潮智能科技有限公司 | Data processing method, device, equipment and storage medium for image compression |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023045257A1 (en) | 2023-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113554574A (en) | Compressed sensing image recovery method, device, equipment and medium | |
| US20160253220A1 (en) | Data center operation | |
| CN109902002B (en) | Generation method and device of combined test case, storage medium and computer equipment | |
| EP3113020A1 (en) | Data processing device and method for processing serial tasks | |
| US9612876B2 (en) | Method and apparatus for estimating a completion time for mapreduce jobs | |
| CN114626503A (en) | Model training method, target detection method, device, electronic device and medium | |
| CN111784000A (en) | Data processing method and device and server | |
| CN110609686A (en) | Data system generation method and device, computer equipment and storage medium | |
| CN109359109B (en) | Data processing method and system based on distributed stream computing | |
| US11557047B2 (en) | Method and apparatus for image processing and computer storage medium | |
| CN112527416A (en) | Task processing method and device, computer equipment and storage medium | |
| KR101643886B1 (en) | System and method for analyzing large-scale high resolution satellite image | |
| Weber et al. | Visual comparison of trace files in vampir | |
| US10909021B2 (en) | Assistance device, design assistance method, and recording medium storing design assistance program | |
| CN113704995B (en) | Non-dominated inequality-based unrelated parallel machine scheduling method and related equipment thereof | |
| CN114549945A (en) | Remote sensing image change detection method and related device | |
| CN115237399A (en) | Method for collecting data, storage medium, processor and engineering vehicle | |
| CN114330719A (en) | Method and electronic equipment for discovering association rule from time sequence chart of events | |
| CN111292223B (en) | Graph calculation processing method and device, electronic equipment and storage medium | |
| CN110083357B (en) | Interface construction method, device, server and storage medium | |
| Li et al. | Hyperspectral video processing on resource-constrained platforms | |
| CN113220646A (en) | Data analysis method and device, computer equipment and storage medium | |
| Raguvir et al. | Analysis of explicit parallelism of image preprocessing algorithms—a case study | |
| CN112579275A (en) | Task execution method, device, equipment and storage medium | |
| CN105095071A (en) | Method, device and system used to obtain scene performance information of application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211026 |
|
| RJ01 | Rejection of invention patent application after publication |